CloudKit Sharing Available in Ensembles 2.5.1

Ensembles 2.5.1 has just been released, and it includes support for CloudKit sharing. You can privately share any Core Data persistent store between one or more iCloud users. The store can be updated by any of the users, and there is no charge to the developer — data is 'billed' to the iCloud account of the user sharing the store.

You can share each persistent store with a single group of iCloud users. This allows for various approaches to sharing. For example, you could share all the data in the app between one group, or you could split your data across multiple persistent stores, and share each one with a different group.

If you own an Ensembles license, you can get the latest version using the download link in your original purchase email. (Contact info@mentalfaculty.com if you have lost it.) More information on how to setup sharing can be found in the CDECloudKitFileSystem.h file.

Dropbox v2 Now Supported in Ensembles

Dropbox recently introduced a new version of their APIs, and announced that the original API is deprecated and will stop working in around a year's time. The Objective-C framework has just arrived, and — as promised — we have added support for this in Ensembles 2. (Support in the open-source Ensembles v1.x will be added at a later date.) 

Currently, you will need access to the master branch of the private repo of Ensembles to use the new class. The Idiomatic sample app has been updated to demonstrate usage of the new backend class.

The new API should be more stable, perhaps faster, and has the advantage that uploads are muted, which means your users will not see lots of notifications with gibberish file names.

Unfortunately, the authentication process is slightly different in the new version. What this means is that your app will need to deleech, and leech again. Assuming you have everything setup right, this should happen fairly automatically, only requiring users to login to Dropbox again. You might decide it is better to wait for a major version upgrade of your app before making the step to Dropbox v2.

Subscribe to Ensembles

Until now, if you purchased an Ensembles package, you would get a year of free updates and support from our team. If you wanted to continue to receive these benefits, you needed to purchase a new package each year.

From now on, you will have another option: a yearly subscription. A subscription gives you ongoing support, and offers the same source code and documentation as the Priority package, including a proprietary Node.js sync server that you can use as the basis of your own sync service.

Even though you get many of the same benefits as with the Priority Package, subscriptions are actually the cheapest option available, at $249. A subscription is automatically renewed on a yearly basis, but the first month is free, and you can cancel it at any time.

Plans to Handle Dropbox Changes

Dropbox recently deprecated their original (v1) API, which is still used inside of Ensembles, and they have announced it will stop working in around a year from now. Where does this leave Ensembles?

For now, apps will continue to work fine, so there is no immediate rush. But within the next few months, we need to start taking steps. We will be moving the Dropbox backend in Ensembles to the v2 API, either using the REST API, or perhaps using an as yet unreleased Objective-C API. We expect to begin that work in August, and it should  be ready a month or two after that.

CloudKit Data Sharing in Ensembles?

Last week at WWDC, CloudKit got the ability to share data between multiple users privately. It has always been possible to share data publicly, and with a bit of URL obfuscation, you could setup quasi-private sharing, but the new CKShare brings much more security.

We often get asked about multiple users sharing data with Ensembles. In fact, Ensembles is completely agnostic in this regard; it doesn't have any concept of users or groups. The Ensembles framework proper just needs to be able to share files between devices, and how those files are accessed and authenticated is purely  a question of backend.

With CKShare we expect to be able to add support for multiple users sharing a persistent store via CloudKit. You will still be responsible for arranging invites and managing users, and will have to create a persistent store for each sharing group, but Ensembles will handle the syncing.

The Core Data Sync Deprecation Sale

Apple announced a number of great improvements to Core Data and CloudKit this week at WWDC, and one significant deprecation: Core Data sync. 

Apple made no mention of the deprecation at all, or any mention of their future plans regarding sync. It is possible that the new generational features are designed to provide the basis of a future, hopefully more robust, Core Data sync. Whatever the case, we probably won't see such a feature for another year at least.

In the meantime, anyone who has adopted Core Data sync is left in limbo. A deprecation is by no means a death sentence — the sync will likely continue to work as it does today for years to come — but there will presumably be no improvements and a degradation over time.

That's why we had decided that now is a good time for a sale, the Core Data Sync Deprecation Sale. If you move to Ensembles now, you not only get the usual support, documentation, and source code, you also get it all at half the standard price. 

Hopefully that takes some of the bite out of Apple's non-announcement.

Update: This sale has now ended.

Dramatic Improvements to CloudKit in Ensembles 2.4

Ensembles 2.4 has been released, and it comes with a whole bunch of improvements to the CloudKit backend. If you are thinking about using iCloud for sync, we highly recommend adopting CloudKit.

The original CloudKit backend was based on queries, to allow for syncing in the public database. We have now added a second possibility: using the private database with a custom zone. With this approach, the queries can be replaced with change fetches, which are much faster because only the most recent changes on the server are returned. 

The new backend is available for download now using the link you receive when purchasing a package.

Important Changes for Dropbox

Dropbox announced earlier in the year that they would be deprecating the Sync and Datastore APIs in October. That deadline is fast approaching.

What does it mean for apps utilizing Ensembles? If you are still using the Sync API, best to move over to the Core API. Users should not experience any major impact, other than perhaps an additional login. The files stored by Ensembles remain in the same locations.

Dropbox have also announced that you need to make minor changes to your app support iOS 9. These changes cannot be made inside of Ensembles — you have to edit the app's Info.plist file. Details are provided here.

Ensembles Now Supports the Dropbox Sync API

Ensembles has had support for Dropbox as a backend since the very beginning, but it was based on the Dropbox Core SDK, which is a thin wrapper around their REST API. 

Ensembles 2 now includes support for a backend based on the Dropbox Sync API. This is more recent to Dropbox's developer offerings, and includes local caching. The new backend has been added to the master branch on GitHub, and will be in the next release of Ensembles 2.

Ensembles 2 can be purchased here, including packages with full source code.

Sync Core Data via CloudKit

Ensembles 2 now supports Apple's new CloudKit framework as a backend for syncing Core Data stores. Anyone who has purchased an Ensembles package which includes source code access can find the new backend class in the GitHub repository. It will also be included in the next release.

CloudKit works at a lower level than iCloud document storage, behaving more like a traditional client-server setup. It should make working with iCloud much more predictable.

Introducing Ensembles 2

It's only been about 4 months since we released Ensembles 1.0, but it's already time for Ensembles 2. We've spent that time rewriting large parts of the framework, with performance and resource usage the focus.

Ensembles 1.x is robust and works great for most apps. But apps with complex models or large data stores could not really make use of the framework, because Ensembles 1.x must be able to load the full store into memory at various times.

With Ensembles 2, our focus was on improving performance, reducing memory usage, and reducing cloud data storage. To give you an idea of the progress made, cloud storage has been reduced by up to 50 times in Ensembles 2. You read that right — many cloud files are only 2% of the size of their Ensembles 1.x counterparts.

Ensembles 2 is not open source. We are making it available with purchases of a package. You can get the binaries for just $179, and full source code starting at $299.

Should you use Ensembles 1.x or Ensembles 2? Ensembles 1.x is still maintained and developed. It's free and open source, and will work well for many apps. If it works for you, there is no need to adopt Ensembles 2.

If you worry about memory usage, or want to minimize cloud storage, Ensembles 2 is probably the way to go. It's a drop-in replacement — the API has been extended, but is fully compatible with Ensembles 1.x — so it's easy to start with Ensembles 1.x and move to Ensembles 2 before shipping.

If you already have a shipping app using Ensembles 1.x, you need to be aware that although Ensembles 2 has an API that is compatible with Ensembles 1.x, it is not binary compatible. In particular, the cloud file storage format is changed, so you should not mix Ensembles 1.x and Ensembles 2 cloud data. The easiest way to migrate is simply to deleech, remove the cloud data, and create a new Ensemble under a different name (e.g. MainStore.v2).

Did CloudKit Sherlock Ensembles?

Amongst the avalanche of announcements from Apple at WWDC last week, one in particular made my heart sink a little: CloudKit. I'm sure there were more than a few people who thought "There goes Ensembles. Sherlock-ed." And for a little while, I had the same fear, though it soon became clear that CloudKit and Ensembles play different roles when it comes to syncing data.

First, let me say, I think CloudKit is awesome. It probably should have been iCloud 1.0 three years ago. Apple have done a great job, and I fully expect this to succeed. It's particularly useful for apps that not only need cloud storage, but also have social aspects.

CloudKit is basically Apple's take on schema-less cloud storage. Think Parse.com or Azure Mobile Services, and you've pretty well grasp CloudKit. You can store data records in the cloud — not just files — and don't have to write any networking code. You can insert records, form relationships, and perform search queries, much like a cloud variant of Core Data (though not as powerful).

As good as it seems to be, there are limitations. CloudKit is not cross platform, so you can forget Android, and there is no web access to the data. But there should be plenty of smaller companies happy just to 'win' the Apple market, so I think it will get adopted. 

It would be easy to assume that CloudKit 'solves' the sync problem, and makes Ensembles redundant. That's not the case. The two address quite different problems.

CloudKit is a means of storing entities in the cloud — it has no local store or even cache. The good news is that you can choose to use any local store API you like, from keyed archiving to SQLite and Core Data. But if you want to sync, you will have to implement local change tracking. CloudKit will not help with this.

That's what Ensembles is good at. Linking CloudKit to Core Data yourself would certainly be possible, but non-trivial. All indications point to Apple integrating the two in the future, perhaps as early as next WWDC, but even if they do, it will still be a couple of years before existing apps could adopt the technology. 

Ensembles is here now, and I've already started adding a CloudKit backend. Initially, it will not take full advantage of the fine-grained record storage of CloudKit, but will use it as file storage. (Interestingly, this is exactly how iCloud Drive will work.)

Further down the track, it may be possible to extend Ensembles to support the CloudKit record storage more natively, doing away with files. And having added CloudKit support, it would be a slam dunk to include native support for Parse, Azure, and maybe even SQL databases. 

And that there is the trump card of Ensembles: if you are using Core Data, we can add backends that become available to you at virtually no cost. No lock in.

Finally, it's worth pointing out that you could complement Ensembles sync with CloudKit social features. Many apps have private user data that just needs to be synced between devices, but they also need some means of sharing a subset of the data between users. Ensembles can handle the complexities of sync, and you can offload the sharing to CloudKit. 

Ensembles 1.0

It's been around a year to the day since I began to dream of a new framework for Core Data sync. Today, I'm proud to announce that I've tagged the repo, pushed the Podfile, and Ensembles 1.0 is a fact.

It all began with my suite of OS X and iOS study apps, Mental Case, which is responsible for 95% of my indie software business. The last few years have been nothing short of a nightmare, as I've struggled to make the transition from reliable peer-to-peer Wi-Fi sync, to the cloud sync that everyone has come to expect.

Eventually the frustration of wrestling existing solutions — including Apple's own offering — led me to the conclusion that if I wanted it done properly, I was probably going to have to do it myself. And so began work on a project initially dubbed 'Syncophant'.

There were many twists and turns along the way. The project was originally envisaged as a Node.js-backed client-server solution, but moved pretty quickly to a file-based, peer-to-peer architecture, which delivers the flexibility to work with virtually any backend service.

Today, Ensembles is a framework worthy of consideration if you want to add sync to a Core Data app. As you would expect from a 1.0 release, it is robust and stable, and is already shipping in a handful of apps.

But 1.0 releases are often light on features, and Ensembles 1.0 is not the right answer for everyone. In particular, on occasion Ensembles must load the whole data store into memory, in much the same way as Core Data's native model version migration. If your app handles large data sets, it may be wise to wait, or — at the very least — run some exploratory tests. Development on Ensembles 2.0 has already begun, and first in line is performance and constraining memory use.

There has been some recent discussion around sync in mobile apps, with some concluding that developing a web service is the only reasonable approach. But there are no right and wrong answers. What works well for the goals of one developer, will be the wrong answer for another.

Each approach has its pros and cons, and you need to carefully weigh your options. If an Android app is definitely on the cards, you are probably better off with a web service. It'll cost you in extra development time, hosting fees, and sleepless nights, but if you have the ambition to develop a cross platform app, you probably also have a team to back you up.

If you have no immediate plans for cross platform support, and just want to add sync in a reasonably cheap and painless way, Ensembles may fit the bill. You can literally have your app syncing with 20–30 lines of code, and support popular services like iCloud and Dropbox.

And there is also the third way: begin with Ensembles, and graduate later to a web service if your app takes off and you decide you need to expand to other platforms. Ensembles is easy to add, and easy to remove. Test the water with Ensembles, and add web services if you end up with a hit, and can afford the team to support it.