Fleece crash during document retrieval about couchbase-lite-ios HOT 12 CLOSED

Closing this issue bc I suspect multithreading issue on my side.

from couchbase-lite-ios.

Reopening bc current docs suggest that multithreading should not be a problem here.

from couchbase-lite-ios.

Is it possible to share the whole crash report

from couchbase-lite-ios.

Attaching lightly edited crash report - I have others that are very similar.
crash_report-2517578-8e3bd82f.txt

from couchbase-lite-ios.

If I want to try reproduce the same, could you please provide below information(preferably with similar code snippets you are using)

1. what steps would you recommend?
2. explain how you are using multiple threads?
3. A sample document with data. (seems like there is a lot nested dictionary > array > dictionary > array >.... )

from couchbase-lite-ios.

1. what steps would you recommend?
   I'm not able to reproduce this crash. The two possible causes I can think of are:

   1. Multithreading (see below)
   2. Some "illegal" data that, although stored successfully, cbl is unable to retrieve.

2. explain how you are using multiple threads?
   The database always gets opened on the main thread. In the four crash reports I've examined, the crash occurs when accessing the db on a different thread.

3. A sample document with data. (seems like there is a lot nested dictionary > array > dictionary > array >.... )
   Attached is a sample StudyHistorySet. Based on the sequence of cbl calls, I'm fairly confident that the sequence is:
   toDictionary(): Document is StudyHistorySet
   toArray(): Array of StudyHistory dicts
   toDictionary(): StudyHistory dict
   toArray(): Array of StudyHistory dicts
   toDictionary(): StudyHistory dict
   toArray(): Array of StudyHistory dicts
   toDictionary(): StudyHistory dict
   toDictionary(): TrialSeries dict

As you can see, it does contain nested dictionaries.

shs10360.json.txt

from couchbase-lite-ios.

Also, the crash always occurs when calling getGroupHistorySets(forContentBundle:). The code is:

func getGroupHistorySets(forContentBundle contentBundle: ContentBundle) -> [StudyHistorySet]  {
    let contentGroups = contentBundle.contentGroups
    let historySets: [StudyHistorySet] = contentGroups.compactMap { fetchStorable(forDocumentType: .studyHistorySet(itemType: .group, itemId: $0.id)) }
    return historySets
}

func fetchStorable<T: Storable>(forDocumentType documentType: StorableDocumentType) -> T? {
    let id = documentType.id(withUserId: userId)
    guard let document = database.document(withID: id) else {
        return nil
    }
    return T.fromStorage(document.toDictionary())
}

from couchbase-lite-ios.

Will be tracking this issue here CBL-3101

from couchbase-lite-ios.

1. From the code snippet shared above, doesn't show any issue: (1) getting the document using the docID, and (2) document converted to dictionary.

The database always gets opened on the main thread. In the four crash reports I've examined, the crash occurs when accessing the db on a different thread.

If you access the database from different thread, it shouldn't have crashed. How are you accessing the db from different thread? Could you share some snippets? May be include the code snippets about the threads as well.

3. Nesting 6-7 levels also not an issue.

4. Also can you share the logic where the JSON is parsed and set into a CBLMutableDocument?

from couchbase-lite-ios.

@jayahariv Thanks for not giving up on this, much appreciated!

First, let me mention that I've released a version of my app that does all its CBL work on a dedicated thread. I have still seen one crash after making this change.

In looking again at the five crash reports I have, I noticed two things:

1. This crash seems to always happen right after app launch
   For example (from crash report):

  Date/Time:       2021-12-01T16:48:04.999Z
  Launch Time:     2021-12-01T16:47:53Z

2. It seems to happen always during the same code sequence. That sequence starts with a method that does all its work on a background queue:

   let userInitiatedQueue = DispatchQueue(label: "contentBundlesViewModel.userInitiated", qos: .userInitiated)

   func theMethodThatCrashes(completion: (Result) -> ()) {
       userInitiatedQueue.async { [weak self] in
          ...
            crash here
          ...
       }
   }

Inside this method, the code path always leads to an init method of another class, and inside init there's a call that hits the db.

3. You asked for "the logic where the JSON is parsed and set into a CBLMutableDocument"

I guess you mean when saving docs to the db? It always goes through this method:

func createStorable<T: Storable>(from storable: T) {
    let id = storable.documentType.id(withUserId: userId)
    let storableDict = storable.toStorage()
    let document = MutableDocument(id: id, data: storableDict)
    document.setString(storable.documentType.name, forKey: Keys.documentType.rawValue)
    document.setString(userId, forKey: Keys.userId.rawValue)
    do {
        try database.saveDocument(document)            
    } catch {
        logger.error("Failed to create storable: \(error)")
    }
}

In the crash, the objects getting retrieved are always StudyHistorySets (which contain StudyHistory objects). This is how these objects get saved:

// Converting StudyHistorySet to dict
func toStorage() -> StorableDictionary {
    var stored: StorableDictionary = [
      .itemId : itemId,
      .itemType : itemType.rawValue,
    ]
    if historiesByIdSet.count > 0 {
        var historyKeys: [[Int]] = []
        var historyValues: [StorableDictionary] = []
        historiesByIdSet.forEach { entry in
            historyKeys.append(Array(entry.key))
            let history = entry.value
            let dict = history.toStorage()
            historyValues.append(dict)
        }
        stored[.historyKeys] = historyKeys
        stored[.historyValues] = historyValues
    }
    return stored
}

// Converting StudyHistory to dict
func toStorage() -> StorableDictionary {
    var stored: StorableDictionary = [
      .itemId : itemId,
      .itemType : itemType.rawValue,
      .isMarkedForRemedialStudy : _isMarkedForRemedialStudy,
      .isMarkedForEarlierReview : _isMarkedForEarlierReview,
      .isMarkedForDelayedReview : isMarkedForDelayedReview,
      .criterionTier : criterionTier.rawValue,
      .numberOfTrials : numberOfTrials,
      .numberOfTrialsSinceLastDueDateSet : numberOfTrialsSinceLastDueDateSet,
      .numberOfSessions : numberOfSessions,
      .numberOfErrors : numberOfErrors
    ]
    addOptional(toDict: &stored, key: .dateDue, value: $dateDue.getStringValue())
    addOptional(toDict: &stored, key: .trialSeries, value: trialSeries?.toStorage())
    addOptional(toDict: &stored, key: .sessionSeries, value: sessionSeries?.toStorage())
    let criterionScoresByTierDict = criterionScoresByTier?.mapEntries { entry in
        return (entry.key.stringValue, entry.value.value) // (criterionTier, clampedValue)
    }
    addOptional(toDict: &stored, key: .criterionScoresByTier, value: criterionScoresByTierDict)
    if let countOfConfusionsByLearnableId = self.countOfConfusionsByLearnableId {
        let (countOfConfusionsKeys, countOfConfusionsValues) = countOfConfusionsByLearnableId.parallelize()
        stored[.countOfConfusionsKeys] = countOfConfusionsKeys
        stored[.countOfConfusionsValues] = countOfConfusionsValues
    }
    if childHistoryMap.count > 0 {
        let values = childHistoryMap.values.map { $0.toStorage() }
        stored[.childHistoryMapValues] = values
    }
    return stored
}

from couchbase-lite-ios.

I have a project attached, which does the similar steps mentioned, but not able to reproduce.

1. Save the shared JSON via main thread.
2. Read the document from userInitiated thread.
3. Convert the document to dictionary via doc->toDictionary()

I can rule out:

• data issue with the attached input JSON.
• save from main thread and different thread to call doc->toDictionary()

Also the above covertion logic(StudyHistorySet/StudyHistory to dict) seems like not depended on Couchbase Lite.

SampleProj.zip

from couchbase-lite-ios.

@philmitchell I will be closing the tracking Jira ticket. We can keep this GH issue open for couple more days, and if you can gather more info which helps us track down the issue. please do post info, I can open a new Jira issue.

from couchbase-lite-ios.