Migration of CoreData to Firestore

Overview

This article explains how to migrate a SQL database into a NoSQL database. Developers who have a fundamental understanding of how both of these databases work, will find this article to be very helpful. CoreData works with relational databases, however Firestore stores data as documents, so converting CoreData to Firestore is not a straight-forward process. Let’s quickly review the terminology for more in-depth comprehension before moving on.

What is CoreData

According to Apple: Use Core Data to save your application’s permanent data for offline use, to cache temporary data, and to add undo functionality to your app on a single device. To sync data across multiple devices in a single iCloud account, Core Data automatically mirrors your schema to a CloudKit container.

What is Firestore

According to Firebase: Firestore is a NoSQL document database that lets you easily store, sync, and query data for your mobile and web apps at global scale. It is a flexible, scalable database for mobile, web, and server development from Firebase and Google Cloud. It keeps your data in sync across client apps through real-time listeners and offers offline support for mobile and web so you can build responsive apps that work regardless of network latency or Internet connectivity. Cloud Firestore also offers seamless integration with other Firebase and Google Cloud products, including Cloud Functions.

Object Graph

An Object graph represents the relationship between two objects through references or pointers. The objects in an object graph can have complex relationships with each other, such as parent-child relationships or many-to-many relationships.

Requirements

• Implement a backup feature that allows the user to export data from CoreData and store it in Firestore.

• The backup process should be initiated by the user and should include all data from all entities within the CoreData stack.

• The exported data should be transformed into a format compatible with Firestore and stored in a designated Firestore collection.

• To resume the backup process in cases like unavailability or a slow internet connection, app termination etc.

Possible Solutions

When faced with the requirement, there are two main options available

• Sync Library

  Sync is a Swift library that simplifies syncing JSON responses with Core Data. Using a convention-over-configuration approach, it maps your data model to infer all the necessary mappings. Its intuitive design makes it easy to use, and you’ll wonder how you managed without it.

  Problems with Sync:

  The application crashes on converting inverse relationships to JSON.

  Reference:

  https://cocoapods.org/pods/Sync

• Custom Solution

  Upon evaluating the Sync Library, it was determined that it did not meet all of the required features and customization needed for the project. Therefore, the decision was made to implement a Custom Solution.

  The custom solution involved defining the Core Data entities to be backed up and mapping them to Firestore collections/documents.

  The steps of implementation have been described in detail below.

Graph Traversal Algorithm

Although the Graph Data Structure and the Object Graph are not exactly identical, they do have several features in common, such as the ability to specify relationships between objects and to traverse relationships using key paths.

So, we can use a Graph Traversal Algorithm to parse the data inside the CoreData.

We have two types of algorithms for this approach.

• BFS: Breadth First Search:

  The BFS algorithm traverses a graph by visiting all the vertices in breadth-first order. It processes all the vertices at the same level before moving to the next level. To keep track of the vertices to visit, the algorithm uses a queue and marks each visited vertex to avoid revisiting it.

• DFS: Depth First Search

  DFS, on the other hand, traverses a graph by visiting all the vertices in depth-first order. It goes as far as possible along each branch before backtracking. The algorithm utilizes a stack to keep track of the vertices to visit and marks each visited vertex to avoid revisiting it.

Conversion from CoreData to Firestore

The use of the DFS algorithm can be advantageous when dealing with large depth CoreData object graphs, as it can be more memory-efficient compared to the BFS.

Here are the steps to be followed.

• Fetch data from CoreData:

  NSFetchedResultsController can be used to manage the results of a CoreData fetch request. The controller provides properties like fetchOffset, returnsObjectsAsFaults, sortDescriptors etc. to get your desired data.

• Keep Track of all the visited nodes / entities.

• Apply Algorithm

  After getting the data, the algorithm needs to be applied to parse the data.

  a. Create a method in the class, let’s name it visitDepth. The method looks like

  func visitDepth(oldManagedObject: [String], _ managedObject: NSManagedObject ) { }

  The oldManagedObject accepts the array of visited entities’ names and the managedObject accepts an Entity. Keep in mind that this method is going to be recursive.

  b. Iterate all the attributes of the entity using For-Loop and store the attribute’s name as key and entity as value in the dictionary. If you don’t know how to fetch all the attributes, use “managedObject.entity.attributesByName.keys“

  c. Now, Perform iteration over all the relationships this entity is having.

  • If the relationship is “to-one” and this entity has not been visited before, then give a recursive call to visitDepth(oldManagedObject, managedObject).

  • If the relationship is “to-one” but the entity has been visited, then update the dictionary as in step b.

  • In case, the relationship is “to-many” and the entity is unvisited, iterate over all these entities and then give a recursive call to the method visitDepth(oldManagedObject, managedObject) again in each iteration.

  • Now, we have the data in the dictionary. We can easily upload this data to the Firestore.

  • In order to continue backup later, save the value of the most recent entity that was uploaded to Firebase in the UserDefaults.

Possible Issues:

• Processes may enter into the state of infinite recursive calls.

• The application may crash due to unmanaged memory leaks.

Our Achievements:

• Data migration from Core Data to Firestore was implemented successfully.

• Feature implemented to enable backup resumption in the event of internet unavailability or application termination without proper backup.

• The feature ensures that data is not lost and that the backup process can be resumed from the point of interruption.

• This feature provides an enhanced user experience by reducing the likelihood of data loss and mitigating the impact of unexpected events on the backup process.

KnowledgeBase:

Graphs and its traversal algorithms
Core Data (CRUD) with Swift 4.2 for Beginners | by Ankur Vekariya | Medium
Step-by-Step Guide to Use Firebase Firestore in an iOS App | by eSparkBiz | Quick Code | Medium