Commit Graph

3558 Commits (fe0306e6be136f90ca12e647611fba12ab5de483)
 

Author SHA1 Message Date
colefranz fe0306e6be issue-2023: move opened event out of selected conversation check
Going through the git history the existing logic of "dont do this if
it's already selected" was just for audio or video QOL enhancements to
not stop playing when the same conversation is selected.
7 years ago
Daniel Gasienica 3ae17528d3
Migrate attachments in background without index (#2208)
- [x] Implement batch migration of attachments without index, i.e. use default
      primary key index on `Message::id`.
- [x] Run attachment in background without index.
- [x] Prepare module for whole database migrations in the future. Once we enable
      that, we have to force (remaining) attachment migration upon startup.
- [x] Run migrations explicitly on startup and remove implicit migrations from
      Backbone models using a placeholder that throws an error.
- [x] `Signal.Debug`: Add support for generating real-world data for
      benchmarking based on contents in `fixtures` folder. Add additional files
      to create a larger variety of test cases, e.g. JPEG, PNG, GIF, MP4, TXT,
      etc. **Test command:**
      ```
      Signal.Debug.createConversation({
        ConversationController,
        WhisperMessage: Whisper.Message,
        numMessages: 100,
      });
      ```
- [x] Minor: Improve error message for `storage.fetch` failures.
- [x] Minor: Use ISO-8601 timestamp for key rotation (helped me debug an issue).
- [x] Update tests to explicitly run migrations.
7 years ago
Daniel Gasienica 8474f3cf7f Use `create*` prefix to clarify curried functions 7 years ago
Daniel Gasienica 6e6b93d917 Run ESLint on `preload.js` 7 years ago
Daniel Gasienica 12cdeee7ec Rename `count` to `numMessagesPerBatch` 7 years ago
Daniel Gasienica 38ac9972e8 Improve termination condition 7 years ago
Daniel Gasienica 5b1a0c45df Replace `getAll` with cursor
This reduces the likelihood of IPC message size error.
7 years ago
Daniel Gasienica ed77378293 Fix bug in termination condition 7 years ago
Daniel Gasienica 9c7afab21b Clear database before message view tests 7 years ago
Daniel Gasienica 9065b647ad Run migrations before clearing database during tests 7 years ago
Daniel Gasienica 01480f5dd2 Improve formatting 7 years ago
Daniel Gasienica 7d0871ee0e Run tests using `yarn` instead of `npm` 7 years ago
Daniel Gasienica 052fb841f6 Allow database name override in migrations 7 years ago
Daniel Gasienica 1e04083813 Support database upgrades in `Database.open` 7 years ago
Daniel Gasienica f50ac1c99e Reduce excessive logging about database status 7 years ago
Daniel Gasienica 77b1f3c505 Replace startup with attachment background migration
Iterates through `Message::id` index.
7 years ago
Daniel Gasienica c853d1bccd Remove whole database migrations
We should enable this when we are willing to run attachment migration
on startup.
7 years ago
Daniel Gasienica 057762806e Add support for attachment background migration without index 7 years ago
Daniel Gasienica 32ac99b439 Prefer ISO date formatting
Previously, this was formatted using locale specific settings. This will allow
easier debugging by matching key rotation timestamps against log timestamps.
7 years ago
Daniel Gasienica 2e4893f4c1 Clarify implications of introducing new migrations 7 years ago
Daniel Gasienica 7413b787b6 Make `migrations` private 7 years ago
Daniel Gasienica b07c66eaa7 Generate attachments from local files
This ensures our benchmarking is closer to real-world usage, e.g. images, video,
etc. that are not compressible.
7 years ago
Daniel Gasienica 4a664bdab8 Add public domain files for generating test data 7 years ago
Daniel Gasienica 064a40544c Fix error message formatting 7 years ago
Daniel Gasienica 442f5d7295 Use `done` to match JavaScript iterators
See: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Iterators_and_Generators
7 years ago
Daniel Gasienica a4f0c5242f Document why batch size is one 7 years ago
Daniel Gasienica d59fdddb43 Improve `storage.fetch` error message 7 years ago
Daniel Gasienica 805031ade8 Conditionally run post-attachment migrations
Introduce placeholder migrations for Backbone models so they never implicitly
run migrations whenever they are `fetch`ed. We prefer to run our migrations
explicitly upon app startup and then let Backbone models be (slightly) dumb(er)
models, without inadvertently triggering migrations.
7 years ago
Daniel Gasienica 887bd83852
Migrate attachments on startup (#2193)
- [x] Run initial Backbone migrations 12–17 upon startup.
- [x] ~~Run attachment migration (without Backbone references to avoid
      migrations): `MessageDataMigrator.processAll`.~~ Disabled in favor of
      background processing without index.
- [x] ~~Run new migrations that can cover entire database (18+).~~
      Not until we have such migrations.
- [x] Add `runMigrations` module that ensures migrations are only run once.
- [x] Add `settings` (`Signal.Settings`) module to interact with our app
      settings (`items` store) using a domain API, e.g.
      `isAttachmentMigrationComplete(…)` vs
      `storage.get('attachmentMigration_isComplete')`. Required to check
      attachment migration status without using Backbone.
- [x] Add `database` (`Signal.Database`) CommonJS module to provide
      `Promise`-based interface to IndexedDB.
- [x] Add `debug` (`Signal.Debug`) module for generating synthetic data to test
      performance of migration.
- [x] Minor: Add `sleep` module for doing promise based sleeps.
- [x] Minor: Extract `wrapDeferred` as CommonJS module named `deferredToPromise`.
7 years ago
Daniel Gasienica bfbeedab5c Temporarily disable post-attachment migration migrations 7 years ago
Daniel Gasienica d9be6a0f94 Destructure Lodash `require`s 7 years ago
Daniel Gasienica b7b6195cfc Extract IIFE into separate function 7 years ago
Daniel Gasienica 11f98474ba Capture how many messages we have to process 7 years ago
Daniel Gasienica 0fdc1140dd Add `Database.getCount` function 7 years ago
Daniel Gasienica 1f8556b049 Remove unused `createCollection` 7 years ago
Daniel Gasienica c67c2a858a Remove Backbone references for attachment migration
It has to run without any other migrations interfering.
7 years ago
Daniel Gasienica 77f8f598de Add `disk-usage.sh` script for testing 7 years ago
Daniel Gasienica 0c40f35623 Document disadvantage of fetching messages without index 7 years ago
Daniel Gasienica a4ecf1a9d6 Define constant after creating idle detector 7 years ago
Daniel Gasienica d3c9de4712 Add debug module
Lets us generate large conversations with large attachments.
7 years ago
Daniel Gasienica 30037e5308 Reduce attachment migration batch size to 1
This prevents ‘Maximum IPC message size exceeded’ due to IDB `getAll` operation.

- https://github.com/zincbase/zincdb/issues/17
- https://cs.chromium.org/chromium/src/content/browser/indexed_db/indexed_db_database.cc?q=%22Maximum+IPC+message+size+exceeded%22&sq=package:chromium&l=1160
7 years ago
Daniel Gasienica ce8fd3d847 Expose `Signal.Settings` module 7 years ago
Daniel Gasienica 02354ce655 Expose `Signal.Database` module 7 years ago
Daniel Gasienica 21147a20a0 Add `sleep` module 7 years ago
Daniel Gasienica 08f73b8420 Remove last processed index after attachment migration 7 years ago
Daniel Gasienica 696a144ab7 Add `settings.deleteItem` 7 years ago
Daniel Gasienica a18e462817 Move migrations to `Signal.Migrations` 7 years ago
Daniel Gasienica 5910f84af4 Remove outdated documentation 7 years ago
Daniel Gasienica efe3cd67fc Allow attachment migration run on higher database version 7 years ago
Daniel Gasienica 921c3dba7c Skip migrations that have already been applied 7 years ago