Commit Graph

367 Commits (a989a1a8186bb8a267f7d79f42523616a1059f63)

Author SHA1 Message Date
Scott Nonnenberg 6628b73911 Only show database log entries if task takes longer than 10ms 7 years ago
Scott Nonnenberg 1755e0adfd Move all remaining stores to SQLCipher 7 years ago
sachaaaaa 4b9fcb98d4 Linting and other housekeeping stuff 7 years ago
Scott Nonnenberg 7aa9031c7f Import: Be resilient to deletions from importObject 7 years ago
Beaudan 4598fd222a Cleaned up newNonce stuff. Now doing target calculation with BigIntegers 7 years ago
Scott Nonnenberg a7d78c0e9b Sealed Sender support
https://signal.org/blog/sealed-sender/
7 years ago
Beaudan d4180b3ca6 Some syntax tidying up, plus fixed bug with message data not being a uint8array after IPC message 7 years ago
Scott Nonnenberg 2060118e85 Remove messages/conversations/unprocessed IndexedDB object stores 7 years ago
Scott Nonnenberg e39c999932 Import: Properly handle import with all empty non-convo stores 7 years ago
Scott Nonnenberg 1e562c8293 Properly handle avatar changes in group update messages 7 years ago
sachaaaaa 1cf8710127
Merge pull request #16 from sachaaaaa/friend_response
Handle prekeys in incoming friend request
7 years ago
sachaaaaa 9370e2b74a Merge branch 'master' of https://github.com/signalapp/Signal-Desktop into development
* 'master' of https://github.com/signalapp/Signal-Desktop: (38 commits)
  v1.17.0
  v1.17.0-beta.4
  Ensure that blue group avatars are preserved in dark theme
  Android theme: Incoming quotes take color from containing message
  Conversations have blue avatar backgrounds if no image provided
  Fix lint errors
  Add ca language
  inboxView: Ensure Conversation exists for our own number
  Ensure that file is not attached if we've filtered it
  v1.17.0-beta.3
  Localization updates
  v1.16.3
  Lint fixes
  Lint fixes
  Better handle large numbers of messages in cache on startup
  Keep object stores after conversations migrate to SQLCipher
  Longer timeout for orphaned file cleanup
  Tests and increase consistency for isFileDangerous calls
  Longer timeout for orphaned file cleanup
  v1.17.0-beta.2
  ...

# Conflicts:
#	background.html
#	package.json
7 years ago
Beaudan dfa8b59bc3 Using let and const properly, updated number to uint function, general cleaning 7 years ago
sachaaaaa 025d13a72b Add keyId index for contact prekeys and allow retrieving prekeys for a specific pubkey and keyid 7 years ago
Beaudan ee57c698d7 Client side proof of work attached to messages
Skeleton calcPoW function that prepares payload for hashingFunction to increment Uint8Array nonce

Util function for converting a number to its Uint8Array representation

Actually perform calculate PoW nonce for every message

Adding bytebuffer as dependency and updated package-lock

Move PoW to new file and execute it in child process

TODO: Move send message logic to callback after child process has finished calculating the PoW

Refactored nonce calculation to function to be awaited

More readable timestamp grab plus converted to seconds

Nonce is now returned as a base64 string and included in request header

Also converted timestamp and ttl to string when including them in the header

Added a couple comments and tidied up some nonce logic
7 years ago
sachaaaaa 886f47b8e7
Prekeys sending: first pass (#12)
* add 'recipient' attribute in prekey database to allow binding a prekey to a recipient pub key

* Attach a PreKeyBundleMessage to the ContentMessage in fallback encryption mode

* Skip generating 100 prekeys upon registration

* Store the signed key signature in database
7 years ago
Scott Nonnenberg 7532f397ea Keep object stores after conversations migrate to SQLCipher 7 years ago
Scott Nonnenberg 8edfbdcee5 Longer timeout for orphaned file cleanup 7 years ago
Scott Nonnenberg ca61c9cb85 Warnings for dangerous files 7 years ago
sachaaaaa 6fe70b9b85 Send outgoing messages as raw bytes (Uint8Array) and moved recipient & ttl to header 7 years ago
Scott Nonnenberg f38410976d Delete conversation external files on deletion 7 years ago
Scott Nonnenberg 2cfbfe477a Better handle large numbers of messages in cache on startup 7 years ago
Scott Nonnenberg a8e12d0771 Add user's avatar to the top left of the screen 7 years ago
Scott Nonnenberg cd60bdd08a Move conversations to SQLCipher 7 years ago
sachaaaaa 33ad509fbf Merge branch 'master' of https://github.com/signalapp/Signal-Desktop into development
# Conflicts:
#	_locales/ar/messages.json
#	_locales/es_419/messages.json
#	_locales/he/messages.json
#	_locales/hi/messages.json
#	_locales/hr/messages.json
#	_locales/kn/messages.json
#	_locales/ko/messages.json
#	_locales/mk/messages.json
#	_locales/sr/messages.json
#	_locales/uk/messages.json
#	_locales/vi/messages.json
#	package.json
7 years ago
Scott Nonnenberg d0c4152d99 backup.js: Remove references to nonexistent i18n strings 7 years ago
sachaaaaa a6b4215b1f changed sendMessage request from PUT to POST 7 years ago
sachaaaaa 8bb7185c7a make loki_message_api::sendMessage use async/await 7 years ago
sachaaaaa bb65115d7d Ensure the unique object id is stored in the object to facilitate deleting it later. 7 years ago
sachaaaaa 2f562ce9e1 Change contact prekeys in db to allow multiple entries for each contact 7 years ago
sachaaaaa 0832806bd9 wip: added helper to create mock contacts, almost got a session working 7 years ago
sachaaaaa 258a89bc21 create contactPreKeys and contactSignedPreKeys in database, with load/store helpers (rerun migrations required) 7 years ago
sachaaaaa e3cbf1caeb rename IndexedDB database to 'loki-messenger' 7 years ago
sachaaaaa b84ff76faf Fix error handling when sending message + clean up in loki_message_api 7 years ago
sachaaaaa 8ada6ce8ad serialize message as base64 7 years ago
sachaaaaa 1e14022e2f Add skeleton for Loki Message API. Wrap data in json (for now?) 7 years ago
sachaaaaa 1866aed432 Accomodate for upcoming signature in websocket connection header 7 years ago
sachaaaaa 187424f84a Change HTTP header from X-Signal-Agent to X-Loki-Messenger-Agent. 7 years ago
Scott Nonnenberg 57d13dad57 data/removeMessage: no need for additional model instantiation 7 years ago
Scott Nonnenberg 02fbea96c0 Move base64 conversions off of the main thread 7 years ago
Scott Nonnenberg 911bc63c67 Move all arrayBuffer<->base64 functions to be async 7 years ago
Scott Nonnenberg 9f920aa35b Re-enable orphaned attachment cleanup 7 years ago
Scott Nonnenberg fedfbed304 Copy quoted message contents into quote on receipt
Also:
  - visually distinguish any reference we couldn't verify on receipt
  - show toast on quote click if we can't scroll to message
  - toast visuals redesigned to match rest of app
7 years ago
Scott Nonnenberg 516a729a45 Additional logging for sql commands in debug mode 7 years ago
Scott Nonnenberg 7983300f4a Use base64 strings for incoming message cache instead of binary 7 years ago
Scott Nonnenberg 8c8475076b Data layer: return null if id not in database, not empty model 7 years ago
Scott Nonnenberg c88df69094 New SQL indices and query tweaks to eliminate table scans
Also: New logging on client side to track performance of db queries
7 years ago
Scott Nonnenberg e086d521f1 Increase the timeout for database updates 7 years ago
Scott Nonnenberg 727925a266 Clean up old messages, better handle errors from sending 7 years ago
Scott Nonnenberg 6e193456f9 Eliminate orphaned external message files on startup
Attachments, visual attachment thumbnails, video attachment screenshots
Quote thumbnails
Contact avatars
7 years ago
Scott Nonnenberg 44dec45995 Ensure that all messages in cache are migrated properly 7 years ago
Scott Nonnenberg a01db40e27 Ensure that a SQL command timeout results in an error 7 years ago
Scott Nonnenberg 22613c8cc4 Set disappearing check timer reliably - on all message saves 7 years ago
Scott Nonnenberg 3961778e53 Fixes for Chrome app import in this new SQLCipher world 7 years ago
Scott Nonnenberg 1d7987108b Show and log progress during the SQLCipher migration 7 years ago
Scott Nonnenberg b1c6a5fe19 When updating message schema pre-SQLCipher, use legacy save 7 years ago
Scott Nonnenberg 0d13d437fe Delete before forced bulk insert to fix broken migrations 7 years ago
Scott Nonnenberg 3105b77475 Migrate to SQLCipher for messages/cache
Quite a few other fixes, including:
  - Sending to contact with no avatar yet (not synced from mobile)
  - Left pane doesn't update quickly or at all on new message
  - Left pane doesn't show sent or error status

Also:
 - Contributing.md: Ensure set of linux dev dependencies is complete
7 years ago
Scott Nonnenberg 02675312c5 Introduce mandatary migration on startup, to minimum version 7 years ago
Scott Nonnenberg f39a96bc76 Move to centralized message/cache data layer
Also, ensure that conversation.messageCollection has nothing in it
unless it has an associated ConversationView.
7 years ago
Scott Nonnenberg 34231168a7 On message delete, ensure that all external files are deleted 7 years ago
Scott Nonnenberg 573a26021b file_input_view: Use makeVideoThumbnail on attach of video 7 years ago
Scott Nonnenberg 5933a34a18 Use window.log in browser context, turn on console eslint rule 7 years ago
Scott Nonnenberg cea38b7bc4 Log clear error if we fail to retrieve debug log upload key 7 years ago
Scott Nonnenberg 2ef2235317 Show error if debug log upload fails 7 years ago
Scott Nonnenberg 4ca674edb2 Remove User-Agent from form to upload debug log 7 years ago
Scott Nonnenberg b159a8d7c7 Update the user agent for debug log requests 7 years ago
Scott Nonnenberg 675e34fc8d New React component: ConversationListItem, installed in left pane
When collecting a conversation's last message, we grab that message's
status as well (if outgoing) and show it.
7 years ago
Scott Nonnenberg 3c69886320 Finish new Message component, integrate into application
Also:
- New schema version 8 with video/image thumbnails, screenshots, sizes
- Upgrade messages not at current schema version when loading messages
  to show in conversation
- New MessageDetail react component
- New ConversationHeader react component
7 years ago
Scott Nonnenberg dc11db92f9 New React component: Message
Also: Use react to render contects on the 'show group members' screen
7 years ago
Scott Nonnenberg ad4387803b New media permission, show dialog when not enabled for voice msg
UI now in separate renderer:
  - the permissions popup
  - settings dialog
  - debug log dialog
  - about window

Couple bug fixes:
  - About Window: Fix 'escape' to close window
  - Remove outdated dist/copy tasks from Gruntfile

Eslintified settings_view.js
7 years ago
Scott Nonnenberg 125c3fafa8
Idle detector: Fix bug preventing it from ever turning off (#2487) 7 years ago
Scott Nonnenberg 8eeaad8e18 Remove unused export/import functionality 7 years ago
Scott Nonnenberg f3bd0cf903 Refactor api.js into web_api, which encapsulates all web access 7 years ago
Scott Nonnenberg 34f423b52c Remove unused code; move signal.js to js/modules for eslint
(noticed as I was doing this that signal.js doesn't get eslint coverage)
7 years ago
Scott Nonnenberg e2d044e02b Filter file scheme; disable http, https and ftp entirely 7 years ago
Scott Nonnenberg 80b069e9b6 Move phone number format/parse into ts/types/PhoneNumber 7 years ago
Scott Nonnenberg aa13a2c6f7 Parse phone numbers into e164 as part of schema upgrade 7 years ago
Scott Nonnenberg d2293d9592 parseAndWriteAvatar: Do shallow copy before omit() call
Otherwise, we get all prototype props/functions, which we can't save in IndexedDB
7 years ago
Daniel Gasienica 3bf8a8966a Notification improvements
- Remove on read, on focus, and on exit.
- Show multi-message notifications like '5 new messages'.
7 years ago
Daniel Gasienica 16bc1d34c6 Message schema 6: Change classification of media and documents
For an easier implementation, we change our original definition of
`initializeAttachmentMetadata`. This means we have to re-run it marked as
version 6 and mark schema version 5 as deprecated as its definition has changed.
7 years ago
Daniel Gasienica 101041f106 Derive `Message.CURRENT_SCHEMA_VERSION` 7 years ago
Scott Nonnenberg 26be658892 Remove 'Contact' from 'Contact.*' properties, clean->parse 7 years ago
Scott Nonnenberg cda326ca45 Use protobuf enum values to eliminate magic numbers 7 years ago
Scott Nonnenberg adbe989949 validateContact: Return error instead of logging 7 years ago
Scott Nonnenberg d0efccd121 A couple renames in contact.js 7 years ago
Scott Nonnenberg 420f81ebcd Move contact migrate code from message.js to contact.js 7 years ago
Scott Nonnenberg 37821e5a1b Wire up all contact behaviors, refactor Contact type/selector 7 years ago
Scott Nonnenberg 3ea3e4e256 Contact sharing: protos and data pipeline
As of this commit: 82b76ccf37
7 years ago
Daniel Gasienica a86056630f Use `Attachment.isVoiceMessage` 7 years ago
Daniel Gasienica 5c8f734e67 Remove newlines from group IDs 7 years ago
Daniel Gasienica a102016ed8 Port `Settings` and `OS` to TypeScript 7 years ago
Scott Nonnenberg 754d65ae2e Let's make it all pretty, shall we?
We missed a couple directories with previous attempts to turn this on
globally: app/ and libtextsecure/

Not to mention files in places we didn't expect: ts files that weren't
in the ts directory!

This turns prettier on for every file we care about (js, ts, tsx, md)
everywhere in the project but for a few key parts.
7 years ago
Daniel Gasienica 6eca5ee940 Document that JPEG auto-orient strips EXIF metadata 7 years ago
Daniel Gasienica 1dd87ad197 Format all source code using Prettier 7 years ago
Daniel Gasienica e4baf27fa7 Increase migration conversion speed by a factor of 3x 7 years ago
Daniel Gasienica d00fb560b0 Avoid exposing `AttachmentTS` 7 years ago
Daniel Gasienica f36f206a01 Use `IndexablePresence` for `hasFileAttachments` and `hasVisualMediaAttachments`
Reduces index size, makes it easier to debug using IndexedDB inspector, and
hopefully improves lookup performance.
7 years ago
Daniel Gasienica 8e3c38d5fe Fix formatting 7 years ago
Daniel Gasienica 146178f977 Update schema documentation 7 years ago
Daniel Gasienica 082ef98a56 Add type definition for `deferredToPromise` 7 years ago
Daniel Gasienica 9d84b2f420 Index messages with attachments using a boolean
When indexing message attachment metadata using numeric indexes such as:

```javascript
{
  conversationId: '+12223334455',
  received_at: 123,
  attachments: […],
  numAttachments: 2,
},
{
  conversationId: '+12223334455',
  received_at: 456,
  attachments: [],
  numAttachments: 0,
}
{
  conversationId: '+12223334455',
  received_at: 789,
  attachments: [],
  numAttachments: 1,
}
```

It creates an index as follows:

```
[conversationId, received_at, numAttachments]
['+12223334455', 123, 2]
['+12223334455', 456, 0]
['+12223334455', 789, 1]
```

This means a query such as…

```
lowerBound: ['+12223334455', 0,                1               ]
upperBound: ['+12223334455', Number.MAX_VALUE, Number.MAX_VALUE]
```

…will return all three original entries because they span the `received_at`
from `0` through `Number.MAX_VALUE`. One workaround is to index booleans using
`1 | undefined` where `1` is included in the index and `undefined` is not, but
that way we lose the ability to query for the `false` value. Instead, we flip
adjust the index to `[conversationId, hasAttachments, received_at]` and can
then query messages with attachments using

```
[conversationId, 1 /* hasAttachments */, 0                /* received_at */]
[conversationId, 1 /* hasAttachments */, Number.MAX_VALUE /* received_at */]
```
7 years ago
Daniel Gasienica 5f220a7b2c Add migration for media gallery indices 7 years ago
Daniel Gasienica d14761087a Add schema version 4: Attachment metadata 7 years ago
Daniel Gasienica df2e6e7864 Port `MIME` module to TypeScript 7 years ago
Daniel Gasienica 6a63e427c8 Use `is` instead of Lodash `is*` 7 years ago
Scott Nonnenberg bf81c3db63
Document Message v4, remove data-free thumbnails 7 years ago
Scott Nonnenberg c02860af5c
Responding to pull request review feedback
- messages.getQuoteObjectUrl: early return
- backup.js: explaining variables for long if statement
- types/messages.js: Log if thumbnail has neither data nor path
- sendmessage.js:
  - remove extraneous logging
  - fix indentation
  - upload attachments and thumbnails in parallel
- preload: don't load fs for tests, just fse
- _conversation.scss: split two selectors into two lines, 0px -> 0
- backup_test.js: use fse.existsSync and comment twoSlashes regex
- network_tests_view_test.js: Comment duplicate assignment to window.getSocketStatus
7 years ago
Scott Nonnenberg a7d44d3344
Backup and end-to-end test! 7 years ago
Scott Nonnenberg 978ff0e2bb
Refactor: Use correct name for sanitized filename in backup.js 7 years ago
Scott Nonnenberg f1ff040842
Finish type definition for _mapQuotedAttachments 7 years ago
Scott Nonnenberg 9ad55c803f
Fix handling attachment thumbnails using thumbnail key 7 years ago
Scott Nonnenberg 054d3887a1
Quotes: The full pipeline into the database
1. MessageReceiver always pulls down thumbnails included in quotes
2. Message.upgradeSchema has a new schema that puts all thumbnails on
   disk just like happens with full attachments.
3. handleDataMessage pipes quote from dataMessage into the final message
   destined for the database
7 years ago
Scott Nonnenberg 21bf02c94d
Fixed examples in Quote.md, rough Android visuals 7 years ago
Daniel Gasienica d7b845326d ESLint auto-fix `link_text` 7 years ago
Daniel Gasienica 9d41b86162 Remove escaping from `linkText`
We leverage jQuery’s HTML escaping in `$.html(…)`.
7 years ago
Daniel Gasienica f04c65088b Fork `link-text` module 7 years ago
Daniel Gasienica 24f4ad53bc Use single quotes for identifiers 7 years ago
Daniel Gasienica 432a6ebd7f Redact file paths with escaped slashes 7 years ago
Daniel Gasienica a8a7525609 Redact stack traces with forward and backslashes 7 years ago
Daniel Gasienica b0da7d965e Redact URL encoded file paths in stack traces 7 years ago
Daniel Gasienica f2c9ccae90 Prefer `is.*` over Lodash `is*` functions 7 years ago
Daniel Gasienica 2ee78ec556 Prefer `path` over `Path` 7 years ago
Scott Nonnenberg c6c3b65bbc
Introduce React, TypeScript, TSLint and React-StyleGuidist (#2219)
Quite a bit of change here.

First, the basics:

- New dependencies were added: react, typescript, tslint, and react-styleguidist
- A new npm script: transpile. It uses typescript to process .tsx files in js/react, putting .js files next to the original file. It's part of the watch functionality of grunt dev as well as the default task run with just grunt (used to build the app prior to release). A lighter-weight to get watch behavior when just working on React components is to run yarn transpile --watch.
- yarn run clean-transpile will remove generated .js files


Style guide via react-styleguidist. Example site: https://react-styleguidist.js.org/examples/basic/

- Start with yarn styleguide
- Component.md files right next to the .tsx file
- jsdoc-style comments are picked up and added to the generated part of the styleguide - the overall summary and a table listing methods and properties of the component
- It has hot-reloading!
- It uses webpack, which means that our app now pulls in webpack though we don't use it to generate anything for the production app.
- I did a bunch of work to enable the use of Backbone views in this context, which will allow us to move smoothly from the old world to the new. First, add all the permutations in the old way, and then slowly start to re-render those same views with React.

A bit of dependency cleanup to enable use in React components:

- moment was moved from our Bower dependencies to our npm dependencies, so it can be used in React components not running in a browser window.
- i18n was moved into the new commonjs format, so it can be used in React components even if window is not available.

Lastly, a bit of Gruntfile cleanup:

- Removal of Chrome App-era modifications of background.js
- Make jshint/jscs watch more targeted, since more and more we'll be using other tools
7 years ago
Scott Nonnenberg 50d4dbaae7
Remove 'use strict' from i18n.js 7 years ago
Scott Nonnenberg 76be13552f
js/i18n.js -> js/modules/i18n.js 7 years ago
Daniel Gasienica 195e9b777c Rename `willHaveAttachmentsSavedOnFileSystemDuringUpgrade`
New name: `willAttachmentsGoToFileSystemOnUpgrade`
7 years ago
Daniel Gasienica 4c4443390a Rename `createImporter` to `createAttachmentDataWriter` 7 years ago
Daniel Gasienica 5ea1a305a6 Clarify error origin 7 years ago
Daniel Gasienica 9dcb64b53c Initialize schema version upon import 7 years ago
Daniel Gasienica cfad983d08 Consistently use `ciphertext` instead of `encrypted` 7 years ago
Daniel Gasienica a61c905146 Impliclity decrypt data if `key` is present 7 years ago
Daniel Gasienica 94c75762b5 Add `Message.createAttachmentLoader` 7 years ago
Daniel Gasienica 43ab320d31 Load attachments before writing them upon export 7 years ago
Daniel Gasienica 7eeabb5b1a Add precondition for `writeAttachment` 7 years ago
Daniel Gasienica 0faaf52b71 Import and upgrade messages upon save 7 years ago
Daniel Gasienica cf7355bd2f Rename `backupToDirectory` to `exportToDirectory`
For symmetry reasons.
7 years ago
Daniel Gasienica bf67254cc5 Add `Message.createImporter` 7 years ago
Daniel Gasienica efd673083d Rename `writeAttachmentData` to `writeNewAttachmentData` 7 years ago
Daniel Gasienica ce8fe0d345 Add `Signal.Startup` module 7 years ago
Daniel Gasienica 8474f3cf7f Use `create*` prefix to clarify curried functions 7 years ago
Daniel Gasienica 12cdeee7ec Rename `count` to `numMessagesPerBatch` 7 years ago
Daniel Gasienica 38ac9972e8 Improve termination condition 7 years ago
Daniel Gasienica 5b1a0c45df Replace `getAll` with cursor
This reduces the likelihood of IPC message size error.
7 years ago
Daniel Gasienica ed77378293 Fix bug in termination condition 7 years ago
Daniel Gasienica 052fb841f6 Allow database name override in migrations 7 years ago
Daniel Gasienica 1e04083813 Support database upgrades in `Database.open` 7 years ago
Daniel Gasienica f50ac1c99e Reduce excessive logging about database status 7 years ago
Daniel Gasienica 77b1f3c505 Replace startup with attachment background migration
Iterates through `Message::id` index.
7 years ago
Daniel Gasienica 057762806e Add support for attachment background migration without index 7 years ago