These were failing because ByteBuffers from the protobufs need to be converted
to ArrayBuffers. Fixed by useing the existing handler in MessageReceiver to
process verified messages from contact sync messages and dispatch them as their
own events, reducing some complexity on the application side.
// FREEBIE
In some cases, due to promise chaining and error propagation, we were
calling registerError more than once for a given error. This would then
cause the overall callback for the send operation to be called with a
partial set of errors, as well as duplicates.
Note: we do need to find a way to attach identityKey to the
OutgoingIdentityKeyError in the case where it comes directly from the
encrypt() instead of our pre-key operations.
FREEBIE
First construct a null message of random size and contents and send it to the
destination. Then include that same padding in the verification sync.
Note that the sync message is additionally randomly padded like all other sync
messages.
This lets Verified sync messages appear the same as normal sync message traffic.
// FREEBIE
Just log em for debugging. Also update the error messge thrown when we get a
content message with no supported properties. It may be empty or may just have
an unrecognized field.
// FREEBIE
This adds a new method to message sender for sending verification sync messages
and a new event to message receiver representing incoming verification sync
messages. Currently the event handler just logs the message.
// FREEBIE
saveIdentityWithAtttributes allows directly setting all properties of an
identity key record. In AccountManager we use it to save our own identity
key after a new registration.
Previously we would remove the existing key first in order to coerce firstUse to
true, but now we can simply set it explicitly, along with a VERIFIED status.
// FREEBIE
Previously we only expected these errors to occur when decrypting prekey
messages, so we need to add handling for them in the case of a normal message.
// FREEBIE
Multiple cases here:
1. setting our own key on registration
1. changing identities from a safety number change dialog
Note that removeIdentityKey runs before saveIdentity, so we'll always end up
with firstUse: true on our own key.
// FREEBIE
We can use the same padded plaintext across multiple numbers or attempts rather
than re-creating it every time we encrypt to a particular number.
// FREEBIE
This test is out of date with respect to the latest signed key rotation rules
implemented in b92dd45 and 536dd7b. Previously we would only keep the last two
signed keys, but now we keep at least three as well as requiring a minimum
retention period. As a result, this test should be failing, but we haven't
noticed because it's not using the `done` parameter to signal its asynchronous
nature.
// FREEBIE
This was supposed to solve the problem of losing group state after a reinstall
by inspecting member registration ids and pre-emptively sending group updates to
those who appear newly re-registered, but it has been unused since 6123c419.
Nowadays there's a protocol for requesting lost group state from other members.
// FREEBIE
There may come a day when we may need to change this url from the server
side. On that day, clients should continue to operate normally. The
service should be able to change attachment server locations without
requiring a client update.
// FREEBIE
Previously, updateNumbers would throw an Error, so the whole group
update was discarded.
Signal-Android handles this the same way in
GroupMessageProcessor.handleGroupUpdate().
Closes#1056
If a user has no groups, no group sync message is sent. This is a normal
result that should result in a success event rather than a timeout.
// FREEBIE
* libsignal.Curve is now a synchronous interface.
* libsignal.Curve.async is now the asynchronous interface.
* Fixes bugs in session management.
// FREEBIE
The getDeviceObjectsForNumber method returns device objects that contain
nothing but the encodedNumber property. Instead, just deal in deviceIds
until a SignalProtocolAddress is constructed to manage both the number
and deviceId.
// FREEBIE
Now that we simply establish and save a session via SessionBuilder
rather than saving the keys to pass in during encrypt, we can stop
caching them in memory in between the identity key check and the
encryption step.
// FREEBIE
Prevent races between encrypt and decrypt calls, and other read/write
operations on the session store by serializing session io ops per
device.
Possible fix for #535
// FREEBIE
Because remote clients will delete all sessions in response to an end
session message, regardless of which device it came from, when our
linked device sends an end session message, we must also end all
sessions with the destination.
This change moves the end session flag processing to processDecrypted,
which is shared between handlers of sent messages, data messages, and
messages which are re-tried after resolving identity conflicts.
// FREEBIE
Help debug bad session errors by logging some envelope info about the
message we are about to decrypt. With this, if there is a decryption
error (e.g., bad mac or no session) it is clear from the logs what
number and device message sent the bad message.
Also log when we send and receive end session messages and when we close
sessions for certain devices.
// FREEBIE
Previously we would delete our own key from the trust store when
re-linking, in case it changed. We can also immediately take one step
further and store the new key. Typically this happens in the course of
requesting sync info from the master device, except in the case of
standalone clients.
Closes#596
// FREEBIE
Sometimes an error is thrown while processing groups from a group sync
message. We still want to fire the groupsync event when we're done
handling all the data, even if some of it was bad.
Similar in function to an xhr request, a textsecure.SyncRequest object
is initialized from a message sender and receiver pair and initiates a
request for sync from the master device. It later fires a success event
when both contacts and groups are done syncing, or a timeout event after
one minute.
// FREEBIE
We should not ack envelope protobufs that fail to decode correctly. If
the server happens to send us such a thing it probably indicates a
protocol mismatch between it and the client, in which case the client
needs to update and re-receive the failed message.
// FREEBIE
This line was never being hit due to the duplicate one above (#L166).
Peeking at the server code shows that 403 is only given in response to
registration attempts, where it does in fact indicate an invalid code.
// FREEBIE
Fix bug where resending due to failed network connection would fail
silently.
Broken in 7b6820 refactor which changed arguments to transmitMessage
// FREEBIE
Previously, libtextsecure would send a sync message automatically
when appropriate. This fails if any recipient has a key conflict
or if our network connection fails mid-send.
Instead, when appropriate, return a the DataMessage encoded as an array
buffer for later syncing. This lets the application choose when to send
it, which we now do after any successful send to a recipient, rather
than after all recipients are successfully sent to.
Eventually we should move the DataMessage protobuf construction and
group sending logic to the application layer entirely, in which case
we wouldn't need libtextsecure to construct the sync message either.
Fixes#408
Fix bug where re-installing would cause sync messages to fail and
produce unresolvable error indicators on messages that were actually
sent.
// FREEBIE
encryptMessageFor now requires an array buffer instead of a protobuf.
By converting the message to an array buffer outside the Promise.all, we
avoid allocating a new copy for each device.
// FREEBIE
Follow up to b0da4910. When inferring membership of the sender in an
unknown group, remember to save and return the group members.
Generally, this should only effect standalone clients unless someone
managed to clear their groups db table, since linked clients get group
info synced at registration.
// FREEBIE
If you're the last member of the group, it doesn't make sense to send
messages to it. Previously, we would wait forever for a callback,
causing a stuck pending state indicator.
// FREEBIE
When we re-register, our deviceId might change, which makes our sessions
are no longer valid since the recipient will see us as a new device.
Fixes#388
Fix a bad loop scope bug in getKeysForNumber by using forEach.
Refactor the initial process of establishing key material for devices
that do not have open sessions.
// FREEBIE
These functions accept an array buffer and extract an AES and MAC key
from it without verifying it has the appropriate length. Ciphertext
messages are similarly dissected. The slice function does not raise an
error on out of bounds accesses but instead returns an empty or
partially-filled array. Empty or short arrays will be passed through to
the window.crypto.subtle API, where they will raise an error. We should
not rely on the Web Crypto API to validate key lengths or for MAC checks
to fail. Instead, validate the lengths of given parameters before
extracting their components.
// FREEBIE
This may increase processing latency a bit, particularly with large
attachments, but will ensure that messages are dispatched in the order
they are received.
It would be nice to enforce ordering on only the dispatch step, so that
we could, for example, decrypt the next websocket message while waiting
for an attachment to download, but that will require a more complicated
refactor. Will stick with the quick fix for now and revisit later.
Fixes#342
// FREEBIE
It's rare that we get in a state where we have a device record without a
session, but we should handle errors gracefully in that case. Catch them
and register them, except for identity key errors which are registered
in handleResult.
// FREEBIE
fixup error handling // FREEBIE
This function dynamically declares a bunch of functions which bind to
its input arguments. Instead, use a new prototypal class to define
these functions within the context of a particular message.
// FREEBIE
Add a pendingMessages object to MessageSender. This object holds
one promise per recipient number. We init this promise with
Promise.resolve(), and chain on promises for message sending, replacing
the previous promise with the newly chained promise each time. If the
current promise resolves and finds that it is still the last promise
in the chain, it removes itself.
Websocket resources should have their keepalive timers reset whenever a
message comes in. This is a nicety that slightly reduces the amount of
traffic we send when actively messaging.
Previously this was handled by MessageReceiver, but it's a bit cleaner
to just have the WebsocketResource add an extra 'message' event handler.
// FREEBIE
These are failing because MockSocket doesn't implement an EventTarget
interface like an actual WebSocket does, so we get an exception when
trying to call addEventListener on it. :(
// FREEBIE
Split into separate encrypt and transmit functions. Let the encryption
function also handle all wire formatting (ie, jsonification and base64
encoding), which simplifes TextSecureServer.sendMessages, removes a
TODO, and lets us save fewer params to make network errors replayable.
// FREEBIE
This ensures that the containing promise is rejected without triggering
the side effects of an uncaught exception, such as causing the debugger
to pause.
// FREEBIE
Previously would fail to register keys by using the wrong username.
The username should be <number>.<deviceid> once we've confirmed our
account and received a deviceId from the server.
// FREEBIE
`tryMessageAgain` is the routine called when re-trying a message that
failed to decrypt due to an IncomingIdentityKeyError. This handling
needs to move to MessageReceiver because it depends on
`processDecrypted` to handle incoming message protos, which depends
on a server instance in order to download attachments.
// FREEBIE
textsecure.MessageSender takes server url and credentials and returns
a message sending interface configured for that server.
Used a wrapper function to insert a TextSecureServer instance into
sendmessage.js code at runtime. This will result in function duplication
between different MessageSender objects, pending further refactoring to
use prototypal inheritence.
// FREEBIE
Following the pattern from previous commit, let the server class accept
a url and login credentials from the caller. Then integrate into
MessageReceiver and AccountManager.
// FREEBIE
Fix inconsistency in error format, where we sometimes get an unexpected
Error object and sometimes get a wrapper object containing an Error.
Also start saving network errors.
// FREEBIE
1. This is nonstandard behavior, not supported by any other clients. It
may help sometimes but will also cause bugs (see 2)
2. iOS doesn't handle group updates with missing fields. all fields must
be populated, and libtextsecure doesn't have any knowledge of the group
name or avatar, so these updates will clobber group state on iOS.
// FREEBIE
This one's been around since forever, but only manifests when someone
leaves the group and comes back. In that case we fail to reinit their
numberRegistrationId object, which causes a npe when we try to send
send them group messages.
Affected parties must ask their fickle friends to leave/join again.
// FREEBIE
Rather than asking for a global target, the message receiver implements
the EventTarget interface itself. It does not expose the dispatchEvent
method, however. This ensures that events can only be triggered from
within the internal MessageReceiver class, which means we no longer need
to namespace them.
// FREEBIE
Let the libtextsecure consumer pass in their own server url, username,
password, and signaling key, as with libtextsecure-java.
Also brings reconnect logic up into the MessageReceiver class, which
is the only place it should apply.
Forgot to bind the socket event handler, and the then() handler should
come before the catch() handler or else it will execute every time the
catch handler executes.
// FREEBIE
Always test connectivity with an http request after a websocket closes,
regardless of what code/error it closed with. If that request succeeds,
automatically reconnect the socket.
// FREEBIE
Start by requesting keys for only the master device, then handle 410 as
needed. Single-device users are the more common case and this strategy
lets us avoid requesting/expending one of our own device keys when
establishing a session with sibling devices.
// FREEBIE
After setting a new identity key as trusted, we retry decryption on all
pending conflicts for that contact. If their identity changed twice in a
row, we can still get a conflict the second time, and should handle it
appropriately.
This new endpoint should always issue a response to a provisioning
socket so if we don't receive one we should assume the connection has
been lost.
Closes#318
By default, automatically disconnect if no response. This is preferable
because we can sometimes lose connectivity without receiving a close
event from the socket, but it's also possible that the endpoint may not
support responses.
// FREEBIE
saveKeysToDeviceObject is the detector of outgoing identity key errors.
Catch these key errors closer to the source by pulling the
getKeysForNumber into the context of sendMessageToDevices, which lets
it access registerError and the message protobuf.
Previously identity key errors would be uncaught if all existing
sessions with a recipient were closed/deleted, since we would
preemptively fetch the new identity key. The old error handling only
kicked in after a 409/410 response from the server when posting a
message encrypted for a stale session.
// FREEBIE
Previously we would convert a bytebuffer to a string, pass it to
libaxolotl where it would be parsed back into a bytebuffer.
Ideally we would just pass the bytebuffer, but it turns out that
libaxolotl's bytebyffer class is identical but separate from
libtextsecure's bytebuffer class. ¯\_(ツ)_/¯
So instead we pass the underlying array buffer, which is handled
more or less the same way as a bytebuffer, and most importantly,
does not involve any copying.
// FREEBIE
We now disconnect ourselves if we don't get the server's response to a
keepalive request within 30s. This way we will eventually disconnect if
the network goes away but the socket is not closed.*
* See code.google.com/p/chromium/issues/detail?id=197841 and
https://stackoverflow.com/questions/11755605/chrome-websocket-connection-not-closed-when-browser-closed
We will then try to reconnect once a minute (See 8a10c96);
Keepalives belong at this level anyway, since the format is defined by
both the websocket resource protocol and our specific server url
structure.
// FREEBIE
The following are equivalent, except that the first is longer and
invokes an extra function call.
```
return new Promise(function(resolve, reject) {
reject(new Error("Unknown Group"));
});
return Promise.reject(new Error("Unknown Group"));
```
The avatar handler was being added to the list of promises too late,
so we were storing the raw avatar protobuf (Long id, bytes key) rather
than the downloaded/decrypted attachment data.
Fixes#280
Protocol and handling is all analogous to contact sync: Multiple
GroupDetails structs are packed into a single attachment blob and parsed
on our end. We don't display the synced groups in the conversation list
until a new message is sent to one of them.
// FREEBIE
Initializing a message receiver opens the socket and starts listening
right away rather than requiring a separate call to connect. The only
other publicly accessible method is to query the socket status.
// FREEBIE
Support for the PLAINTEXT message type is not present in the latest
protobuf definitions. Leaving it out for now since we don't have any use
case for it currently.
Update protobuf definitions and refactor message receive and decrypt
codepath to support new protocol, including various flavors of sync
messages (sent messages, contacts, and groups).
Also cleans up background.js and lets libtextsecure internalize
textsecure.processDecrypted and ensure that it is called before handing
DataMessages off to the application.
The Envelope structure now has a generic content field and a
legacyMessage field for backwards compatibility. We'll send outgoing
messages as legacy messages, and sync messages as "content" while
continuing to support both legacy and non-legacy messages on the receive
side until old clients have a chance to transition.
* Session records are now opaque strings, so treat them that way:
- no more cross checking identity key and session records
- Move hasOpenSession to axolotl wrapper
- Remote registration ids must be fetched async'ly via protocol wrapper
* Implement async AxolotlStore using textsecure.storage
* Add some db stores and move prekeys and signed keys to indexeddb
* Add storage tests
* Rename identityKey storage key from libaxolotl25519KeyidentityKey to
simply identityKey, since it's no longer hardcoded in libaxolotl
* Rework registration and key-generation, keeping logic in libtextsecure
and rendering in options.js.
* Remove key_worker since workers are handled at the libaxolotl level
now
Encapsulate the websocket resources and socket setup process in a
friendly OO class. The MessageReceiver constructor expects an instance
of EventTarget on which to fire message events asynchronously. The
provider of the EventTarget can then add/remove listeners as desired.
Ground work for a smoother registration flow. Overall UX still needs
some polish but at least now we can have a progress gif or animation or
whatever. Also adds the phonenumber-confirmation step as a simple alert
box, which will be replaced with a nice dialogue in a later commit.
We'd like to live in a world where we can retry all the pending
conflicts in a conversation as a batch, which means we don't want to
wipe the identity key before processing each message. Thus, remove that
step from these handlers and encapsulate in a method on the conversation
model.
Ensure that both tryAgain functions return promises, allowing the
application to take appropriate action in the result of success or
failure. This lets us remove all dependency from libtextsecure on
app-level constructs like message objects/ids and the `extenion.trigger`
function.
Corresponding frontend changes to follow in another commit.
1. Return the value returned by the registered function, to expose the
underlying promise to the caller.
2. Stop accepting extra arguments to the replay function. The caller
should be able to do what they want with the returned promise instead.
3. Add a timestamp argument to the outgoing case, needed to re-try
sending a message.
Let libaxolotl throw a generic error instead of a replayable error, and
add an helper function in libtextsecure's axolotl_wrapper to catch and
convert from the generic error to the replayable one. This allows the
ReplayableError to remain a libtextsecure-level concept only.
Somewhat unrelatedly, but nearby, fix some whitespace and add missing
semicolon.
Now with actual malloc/free implementations. Had to drop back to -O1
optimization because the ed25519 signature test broke with -O2. :(
Closes#153
The toolchain install and build process, for reference:
```
wget https://s3.amazonaws.com/mozilla-games/emscripten/releases/emsdk-portable.tar.gz
tar xvfz emsdk-portable.tar.gz
cd emsdk-portable
./emsdk update
./emsdk install latest
./emsdk activate latest
source ./emsdk_env.sh
cd ../TextSecure-Browser
grunt build
```
Turns out we can get ABNORMAL_CODE (1006) for disconnects where (for
instance) we pause the background page too long. However, in these cases
there is no preceeding ErrorEvent. In contrast, when we have bad
authentication credentials, there is an ErrorEvent. Thus, this change
ensures that we only reconnect if there was no Error.
Closes#173
Previously, in the event of a failed websocket auth, we would attempt to
reconnect once a second ad infinitum. This changeset ensures that we
only reconnect automatically if the socket closed 'normally' as
indicated by the code on the socket's CloseEvent. Otherwise, show a
'Websocket closed' error on the inbox view.
Ideally we would show a more contextual error (ie, 'Unauthorized'), but
unfortunately the actual server response code is not available to our
code. It can be observed in the console output from the background page,
but programmatically, we only receive the WebSocket CloseEvent codes
listed here:
https://developer.mozilla.org/en-US/docs/Web/API/CloseEvent#Status_codes
The websocket error message is displayed by a normally-hidden but ever
present socket status element. Clicking this element will immediately
refresh the background page, which will try again to open the websocket
connection.
This is a whitespace-only change. The diff is best viewed
with the `--patience` flag.
Some component library sources include trailing whitespace and carriage
returns which were excluded from these compiled files, (but somehow
still included in the final libtextsecure build). Although superfluous,
excluding these characters from these files means that I end up with
spurious changes whenever I rebuild them.
When sending an constrct a copy of the PushMessageContent protobuf, add
a SyncMessageContext, and send it to ourselves. Do this for all kinds of
group messages, and individual text/media messages, but not closeSession
messages as the latter are device-specific.
Do not sync messages if we are the primary device, which should only be
supported in development. Normal web clients must be paired with a
android or ios master device, and even in dev, a primary/standalone web
client does not support linking additional devices.
Scott Nonnenberg
lilia
haffenloher
Karel Bilek
Matt Corallo
codedust
Matt Corallo