V8.9.3 Adds FileDownloader().start() to streamline the sequence of st…

…art related calls, and rescheduleKilledTasks; removes upload range limit

* Adds `start` which ensures the various start-up calls are executed in the correct order. Use this instead of calling `trackTasks`, `resumeFromBackground` and `rescheduleKilledTasks` separately
* Adds `rescheduleKilledTasks` which will compare enqueued/running tasks in the database with those active in the downloader, and reschedules those that have been killed by the user
* [iOS] Removes limit on range of partial file uploads using the Range header (was 2GB)

CHANGELOG.md

@@ -1,10 +1,15 @@
+## 8.9.3
+* Adds `start` which ensures the various start-up calls are executed in the correct order. Use this instead of calling `trackTasks`, `resumeFromBackground` and `rescheduleKilledTasks` separately
+* Adds `rescheduleKilledTasks` which will compare enqueued/running tasks in the database with those active in the downloader, and reschedules those that have been killed by the user
+* [iOS] Removes limit on range of partial file uploads using the Range header (was 2GB)
+
 ## 8.9.2
 * Upgraded minimum Dart SDK to 3.5.0 / Flutter SDK 3.24.0 to stay in sync with dependency updates
-* Fix bug when uploading files greater than 2GB, that was introduced in V8.9.0
+* [Android] Fix bug when uploading files greater than 2GB, that was introduced in V8.9.0
 
 ## 8.9.1
 * [iOS] Adds Privacy Manifest
-* [iOS] Adds support for Swift Package Manager
+* [iOS] Adds support for Swift Package Manager and defaults the example app to using it
 
 ## 8.9.0
 * Adds `options` field to Task, which take a `TaskOptions` object to configure less common task specific options - currently `onTaskStart`, `onTaskFinished` and `auth`

README.md

@@ -87,6 +87,8 @@ FileDownloader().updates.listen((update) {
       }
     });
 
+FileDownloader().start(); // activates the database and ensures proper restart after suspend/kill
+
 // Next, enqueue tasks to kick off background downloads, e.g.
 final successfullyEnqueued = await FileDownloader().enqueue(DownloadTask(
                                 url: 'https://google.com',
@@ -357,9 +359,10 @@ To ensure your callbacks or listener capture events that may have happened when
 In summary, to track your tasks persistently, follow these steps in order, immediately after app startup:
 1. If using a non-default `PersistentStorage` backend, initialize with `FileDownloader(persistentStorage: MyPersistentStorage())` and wait for the initialization to complete by calling `await FileDownloader().ready` (see [using the database](#using-the-database-to-track-tasks) for details on `PersistentStorage`).
 2. Register an event listener or callback(s) to process status and progress updates
-3. Call `await FileDownloader().trackTasks()` if you want to track the tasks in a persistent database
-4. Call `await FileDownloader().resumeFromBackground()` to ensure events that happened while your app was in the background are processed
-5. If you are tracking tasks in the database, after ~5 seconds, call `await FileDownloader().rescheduleMissingTasks()` to reschedule tasks that are in the database as `enqueued` or `running` yet are not enqueued or running on the native side. These tasks have been "lost", most likely because the user killed your app (which kills tasks on the native side without warning)
+3. Call `await FileDownloader().start()` to execute the following calls in the correct order (or call these manually):
+   a. Call `await FileDownloader().trackTasks()` if you want to track the tasks in a persistent database
+   b. Call `await FileDownloader().resumeFromBackground()` to ensure events that happened while your app was in the background are processed
+   c. If you are tracking tasks in the database, after ~5 seconds, call `await FileDownloader().rescheduleKilledTasks()` to reschedule tasks that are in the database as `enqueued` or `running` yet are not enqueued or running on the native side. These tasks have been "lost", most likely because the user killed your app (which kills tasks on the native side without warning)
 
 The rest of this section details [event listeners](#using-an-event-listener), [callbacks](#using-callbacks) and the [database](#using-the-database-to-track-tasks) in detail.
 
@@ -427,7 +430,7 @@ You can unregister callbacks using `FileDownloader().unregisterCallbacks()`.
 
 ### Using the database to track Tasks
 
-To keep track of the status and progress of all tasks, even after they have completed, activate tracking by calling `trackTasks()` and use the `database` field to query and retrieve the [TaskRecord](https://pub.dev/documentation/background_downloader/latest/background_downloader/TaskRecord-class.html) entries stored. For example:
+To keep track of the status and progress of all tasks, even after they have completed, activate tracking by calling `trackTasks()` (or calling `FileDownloader().start()` with `doTrackTasks` set to true - the default) and use the `database` field to query and retrieve the [TaskRecord](https://pub.dev/documentation/background_downloader/latest/background_downloader/TaskRecord-class.html) entries stored. For example:
 ```dart
 // at app startup, after registering listener or callback, start tracking
 await FileDownloader().trackTasks();
@@ -449,7 +452,7 @@ print('TaskId ${record.taskId} with task ${record.task} has '
 You can interact with the `database` using `allRecords`, `allRecordsOlderThan`, `recordForId`,`deleteAllRecords`,
 `deleteRecordWithId` etc. If you only want to track tasks in a specific [group](#grouping-tasks), call `trackTasksInGroup` instead.
 
-If a user kills your app (e.g. by swiping it away in the app tray) then tasks that are running (natively) are killed, and no indication is given to your application. This cannot be avoided. To guard for this, upon app startup you can ask the downloader to reschedule missing tasks, i.e. tasks that show up as `enqueued` or `running` in the database, yet are not enqueued or running on the native side. Method `rescheduleMissingTasks` returns a record with two lists, 1) successfully rescheduled tasks and 2) tasks that failed to reschedule. Together, those are the missing tasks. Reschedule missing tasks a few seconds after you have called `resumeFromBackground`, as that gives the downloader time to processes updates that may have happened while the app was suspended.
+If a user kills your app (e.g. by swiping it away in the app tray) then tasks that are running (natively) are killed, and no indication is given to your application. This cannot be avoided. To guard for this, upon app startup you can ask the downloader to reschedule killed tasks, i.e. tasks that show up as `enqueued` or `running` in the database, yet are not enqueued or running on the native side. Method `rescheduleKilledTasks` returns a record with two lists, 1) successfully rescheduled tasks and 2) tasks that failed to reschedule. Together, those are the missing tasks. Reschedule missing tasks a few seconds after you have called `resumeFromBackground`, as that gives the downloader time to processes updates that may have happened while the app was suspended, or call `FileDownloader().start()` with `doRescheduleKilledTasks` set to true (the default).
 
 By default, the downloader uses a modified version of the [localstore](https://pub.dev/packages/localstore) package to store the `TaskRecord` and other objects. To use a different persistent storage solution, create a class that implements the [PersistentStorage](https://pub.dev/documentation/background_downloader/latest/background_downloader/PersistentStorage-class.html) interface, and initialize the downloader by calling `FileDownloader(persistentStorage: MyPersistentStorage())` as the first use of the `FileDownloader`.
 

example/integration_test/database_test.dart

@@ -132,16 +132,16 @@ void main() {
   test('rescheduleMissingTasks', () async {
     expect(await FileDownloader().allTasks(), isEmpty);
     // without task tracking activated, throws assertionError
-    expect(() async => await FileDownloader().rescheduleMissingTasks(),
+    expect(() async => await FileDownloader().rescheduleKilledTasks(),
         throwsAssertionError);
     await FileDownloader().trackTasks();
     // test empty
-    final result = await FileDownloader().rescheduleMissingTasks();
+    final result = await FileDownloader().rescheduleKilledTasks();
     expect(result.$1, isEmpty);
     expect(result.$2, isEmpty);
     // add a record to the database that is not enqueued
     await FileDownloader().database.updateRecord(record);
-    final result2 = await FileDownloader().rescheduleMissingTasks();
+    final result2 = await FileDownloader().rescheduleKilledTasks();
     expect(result2.$1.length, equals(1));
     expect(result2.$2, isEmpty);
     expect(result2.$1.first.taskId, equals(task.taskId));
@@ -152,7 +152,7 @@ void main() {
     // add a record to the database that is also enqueued
     expect(await FileDownloader().enqueue(task2), isTrue);
     expect(await FileDownloader().database.allRecords(), isNotEmpty);
-    final result3 = await FileDownloader().rescheduleMissingTasks();
+    final result3 = await FileDownloader().rescheduleKilledTasks();
     expect(result3.$1, isEmpty);
     expect(result3.$2, isEmpty);
   });

example/lib/main.dart

@@ -115,6 +115,11 @@ class _MyAppState extends State<MyApp> {
           progressUpdateStream.add(update); // pass on to widget for indicator
       }
     });
+    // Start the FileDownloader. Default start means database tracking and
+    // proper handling of events that happened while the app was suspended,
+    // and rescheduling of tasks that were killed by the user.
+    // Start behavior can be configured with parameters
+    FileDownloader().start();
   }
 
   /// Process the user tapping on a notification by printing a message

ios/background_downloader/Sources/background_downloader/Helpers.swift

@@ -18,13 +18,13 @@ extension URL {
             return appendingPathComponent(component, isDirectory: isDirectory)
         }
     }
-
+    
     /// Excludes URL from backup
     mutating func setCloudBackup(exclude: Bool) throws {
-            var resource = URLResourceValues()
-            resource.isExcludedFromBackup = exclude
-            try self.setResourceValues(resource)
-        }
+        var resource = URLResourceValues()
+        resource.isExcludedFromBackup = exclude
+        try self.setResourceValues(resource)
+    }
 }
 
 /// Returns the task's URL if it can be parsed, otherwise null
@@ -59,73 +59,52 @@ func lowerCasedStringStringMap(_ map: [AnyHashable: Any]?) -> [String: String]?
 ///
 /// Returns the URL of the temp file, or nil if there was a problem (problem is logged)
 func createTempFileWithRange(from fileURL: URL, start: UInt64, contentLength: UInt64) -> URL? {
-    if contentLength > UInt64(Int.max) {
-        os_log("Content length for partial upload cannot exceed %d bytes", log: log, type: .info, Int.max)
-        return nil
-    }
     let fileManager = FileManager.default
     let tempDir = fileManager.temporaryDirectory
     let tempFileURL = tempDir.appendingPathComponent(UUID().uuidString) // Create a unique temporary file
 
     // Create the temporary file
     fileManager.createFile(atPath: tempFileURL.path, contents: nil, attributes: nil)
-
-    guard let inputStream = InputStream(url: fileURL) else {
-        os_log("Cannot create input stream for partial upload temporary file creation", log: log, type: .info)
+    guard let inputStream = InputStream(url: fileURL),
+          let outputStream = OutputStream(toFileAtPath: tempFileURL.path, append: false) else {
+        os_log("Cannot create input or output stream for partial upload temporary file creation", log: log, type: .error)
         return nil
     }
-
-    guard let outputStream = OutputStream(toFileAtPath: tempFileURL.path, append: false) else {
-        os_log("Cannot create output stream for partial upload temporary file creation", log: log, type: .info)
-        return nil
-    }
-
     inputStream.open()
     outputStream.open()
-
     defer {
         inputStream.close()
         outputStream.close()
     }
-
     let chunkSize = 1024 * 1024 // 1MB chunks
     let buffer = UnsafeMutablePointer<UInt8>.allocate(capacity: chunkSize)
-        defer { buffer.deallocate() }
-
+    defer { buffer.deallocate() }
     var remainingBytes = contentLength
     var totalRead: UInt64 = 0
-
+    
     // Seek to the start position
     while totalRead < start {
         let seekBytes = min(chunkSize, Int(start - totalRead))
         let bytesRead = inputStream.read(buffer, maxLength: seekBytes)
         if bytesRead < 0 {
-            os_log("Cannot read data up to desired start from original file for partial upload temporary file creation", log: log, type: .info)
+            os_log("Cannot read data up to desired start from original file for partial upload temporary file creation", log: log, type: .error)
             return nil
         }
         if bytesRead == 0 { break } // EOF
         totalRead += UInt64(bytesRead)
     }
-
-    // Now read only the required range
+    
+    // Read only the required range
     while remainingBytes > 0 {
         let bytesToRead = min(chunkSize, Int(remainingBytes))
         let bytesRead = inputStream.read(buffer, maxLength: bytesToRead)
-
-        if bytesRead < 0 {
-            os_log("Cannot read data from original file for partial upload temporary file creation", log: log, type: .info)
-            return nil
-        }
-        if bytesRead == 0 { break } // EOF
-
+        if bytesRead <= 0 { break } // EOF
         let bytesWritten = outputStream.write(buffer, maxLength: bytesRead)
         if bytesWritten < 0 {
             os_log("Cannot write data to new temporary file for partial upload", log: log, type: .error)
             return nil
         }
-
         remainingBytes -= UInt64(bytesWritten)
     }
-
     return tempFileURL
 }

lib/src/file_downloader.dart

@@ -488,6 +488,34 @@ interface class FileDownloader {
   /// the status of tasks, use a [TaskStatusCallback]
   Future<Task?> taskForId(String taskId) => _downloader.taskForId(taskId);
 
+  /// Convenience start method for using the database. Must be called AFTER
+  /// registering update callbacks or listener.
+  ///
+  /// Calls in order:
+  /// [trackTasks] to start database tracking, with [markDownloadedComplete] to
+  ///     ensure fully downloaded tasks are marked as complete in the database
+  /// [resumeFromBackground] to fetch status and progress updates that may have
+  ///     happened while the app was suspended
+  /// [rescheduleKilledTasks] (after a 5 second delay) to ensure tasks that were
+  ///     killed by the user are rescheduled
+  ///
+  /// [doTrackTasks] and [doRescheduleKilledTasks] can be set to false to skip
+  /// that step.  [resumeFromBackground] is always called.
+  Future<void> start(
+      {bool doTrackTasks = true,
+      bool markDownloadedComplete = true,
+      bool doRescheduleKilledTasks = true}) async {
+    if (doTrackTasks) {
+      await FileDownloader()
+          .trackTasks(markDownloadedComplete: markDownloadedComplete);
+      if (doRescheduleKilledTasks) {
+        Timer(const Duration(seconds: 5),
+            () => FileDownloader().rescheduleKilledTasks());
+      }
+    }
+    await FileDownloader().resumeFromBackground();
+  }
+
   /// Activate tracking for tasks in this [group]
   ///
   /// All subsequent tasks in this group will be recorded in persistent storage.
@@ -559,10 +587,10 @@ interface class FileDownloader {
   ///
   /// Throws assertion error if you are not currently tracking tasks, as that
   /// makes this function a no-op that always returns empty lists.
-  Future<(List<Task>, List<Task>)> rescheduleMissingTasks() async {
+  Future<(List<Task>, List<Task>)> rescheduleKilledTasks() async {
     assert(
         _downloader.isTrackingTasks,
-        'rescheduleMissingTasks should only be called if you are tracking tasks. '
+        'rescheduleKilledTasks should only be called if you are tracking tasks. '
         'Did you call trackTasks or trackTasksInGroup?');
     final databaseTasks = (await database.allRecords())
         .where((record) => const [

pubspec.yaml

@@ -1,7 +1,7 @@
 name: background_downloader
 description: A multi-platform background file downloader and uploader. Define the task, enqueue and monitor progress
 
-version: 8.9.2
+version: 8.9.3
 repository: https://github.com/781flyingdutchman/background_downloader
 
 environment: