Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
In terms of Spec Core Team MSC focus for this week, we're switching from QR Code verification to key verification additions. Otherwise things remain the same: MSC2844 (global versioning), MSC2366 (ready and done in key verification) and MSC2790 (modal widgets).
On behalf of the Americans on the Synapse team: Happy Thanksgiving! This past week saw us land some improvements to the OpenID Connect SSO provider, bringing it closer to parity with the SAML provider. In particular, #8801 made it possible to gracefully handle username collisions when creating a new account over SSO.
We've also merged a pull request (#8785) from @dklimpel which removes deprecated paths for the Synapse Admin APIs. As of the next Synapse release, all Admin APIs will be accessibly only under the /_synapse/admin/v1 path prefix.
Otherwise, much of our time has been spent on getting pull requests like #6739, implementing MSC2403: Add "knock" feature, ready to merge. It's coming to a Synapse release soon!
Speaking of releases, we're looking forward to cutting a first release candidate of 1.24.0 early next week; keep your eyes peeled for that!
I'd be remiss in not mentioning that we started the week by disclosing CVE-2020-26890, which is a denial of service affecting Synapse versions prior to 1.20.0. If you've not upgraded, please do.
Lots of features added this week including Gitter threaded conversations and Matrix reply bridging, image/file uploads, transforming mentions so notifications flow, and better looking flairs!
We'll be working through some of the bridge oddities and hoping to open up the bridge to all public rooms on Gitter soon. You can track our full progress from the GitLab epic for the native Gitter-Matrix bridge.
The MX-Puppet-VK bridge for VK (Vkontakte) social network via this commit have added initial support for User tokens. This means that we now can use it for puppeting the real user account, to automatically bridge all "VK Messenger" private dialogs and groups into Matrix. Previously bridge worked only with VK community tokens as bot, to bridge only conversation with specific communities.
The matrix-sms-bridge can finally be used with an Android smartphone as SMS-Gateway instead of gammu and a modem. @rebekka:imbitbu.de developed the app android-sms-gateway-server, that the bridge uses to send and receive sms messages. The new bridge version is used in combination with the app in production (medical care centre) for a week now. We have a throughput of 50-100 SMS message a day. The update is also integrated in matrix-docker-ansible-deploy.
Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
The new completer is now on master. Please try it out and report any issues you may find. It now actually inserts a user link, keeps a per room message history and should look quite a bit nicer and be faster, when completing emojis or user names.
There were also quite a few cleanups, most of them done by Kirillpt! Some spacing here and there, saturation changes, the login page should now be less confusing, when you don't enter a valid mxid and toggles should now not get stuck in confusing intermediate states. There is also a russian translation now and some VOIP fixes. Pretty busy week, but I like it!
FluffyChat 0.23.0 is out now with password recovery options and a lot of minor fixes and UX improvements. Now also available on Flathub!! 🙂 Learn more at: https://fluffychat.im
On mobile, we’re busy adding better suggestions to empty/starting views of screens, improving searching for contacts, improving adding other users by QR code scanning, and improving deep link integrations on Android. Expect releases for iOS & Android soon!
Spaces
Spaces UI designs & user research testing are underway, with initial development for the Element Web client kicked off too. We've been doing research & testing in private to avoid advance bias from the people we're testing with, but hope to share things publicly soon.
Social login
Social login is in review for Element Web, with synapse & mobile support hot on the tails for next week prior to releasing.
VoIP
We’ve been focusing on implementing UI for call holding, and general UI polish across the VoIP experience, to be released soon!
Element Web 1.7.14 was released this week, which includes:
Several tweaks and improvements to the room list filter
Improved registration based on user feedback
Improved invite / create DM flow.
Future releases will no longer publish element-web to NPM, and the package is marked as deprecated
Renamed build artifacts from riot-.tar.gz to element-.tar.gz
Experimental LaTeX maths rendering support is available via a labs flag on develop. Thanks to akissinger, thosgood, uhoreg, and others who have worked on various pieces of this puzzle!
Element iOS
Element iOS 1.1.1 has been released:
It contains contains several major bug fixes, including end-to-end encryption bug fixes.
Full history available on GitHub viewing the 1.1.1 & 1.1.0 tags.
Element Android
The Element Android 1.0.11 release is pending:
We're working through final bug fixes.
Separately, we’re also working on adding missing features to the client like; managing room aliases, room history visibility, join rules, etc.
In the SDK, the migration of the services API from MatrixCallback to coroutines is going well, tracked here.
In addition to boring stuff like being able to persist events (hopefully) right, so we don't have to initial-sync every launch, I've been making a start towards multi-account support!
You can sign in as multiple users and switch between the "active" user you want to send messages as - it's a bit flaky for now, esp. when scrolling up to read past messages, but hey it's a start I guess
Asked if they were using an SDK:
Preeetty much doing it from scratch, though I am loosely basing the matrix-api side of it on Ruma
We gained a new contributor, kiwidu38, who added the server-default push rules, paving the way to Conduit supporting push notifications.
I improved compatibility with some commonly-broken DAG events and added rudimentary support for message edits to help Fractal finish their move onto matrix-rust-sdk.
I have not read this series so I must trust that Samuel is saying nice things! Anyway it seems like a useful resource for telling kameraden about getting started with Matrix.
Today there is a German article about Famedly on Gründerszene -> https://www.gruenderszene.de/health/famedly-kommunikation-app-gesundheitswesen. Famedly is developing a solution based on Matrix to enable healthcare providers to communicate securely with each other and to facilitate intersectoral processes such as patient transfers or the integration of IomT devices. We have also almost completed a successful financing round and hope to support the Matrix ecosystem more and more. It would be so cool to use FluffyChat, Element, or a Matrix Messenger of your choice to start a video consultation with the doctor of your choice, or to be integrated into existing clinical communication flows concerning you!
Congratulations to Famedly on the progress they're making! If you're ready to graduate from TLAs (Three-Letter-Abbreviations) to the next stage, you'll be please to learn "IomT", which means "Internet of medical Things".
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
In terms of Spec Core Team MSC focus for this week, we're sticking with the same three MSCs: MSC2844 (global versioning), MSC1544 (QR code verification) and MSC2790 (modal widgets). MSC2844 in particular I personally found quite interesting.
We started out this week by releasing Dendrite 0.3.0 and then ended the week with a bug-fix Dendrite 0.3.1 release.
I'd like to say thank you especially to our community contributors who have been adding new features and filling gaps!
Changes this week include:
Memory optimisation by reference passing, significantly reducing the number of allocations and duplications in memory
A concurrency bug has been fixed in the federation API that could cause Dendrite to crash
A hook API has been added for experimental MSCs, with an early implementation of MSC2836
Forgetting rooms is now supported (thanks to S7evinK)
The last seen timestamp and IP address are now updated automatically when calling /sync
The last seen timestamp and IP address are now reported in /_matrix/client/r0/devices (thanks to alexkursell)
An optional configuration option sync_api.real_ip_header has been added for specifying which HTTP header contains the real client IP address (for if Dendrite is running behind a reverse HTTP proxy)
Partial implementation of /_matrix/client/r0/admin/whois (thanks to DavidSpenler)
The error when registering a username with invalid characters has been corrected (thanks to bodqhrohro)
The -version command line flag has been added (thanks to S7evinK)
Backfilling should now work properly on rooms with world-readable history visibility (thanks to MayeulC)
Some more client event fields have been marked as omit-when-empty (thanks to S7evinK)
The build.sh script has been updated to work properly on all POSIX platforms (thanks to felix)
Spec compliance is unchanged, although some Synapse-specific tests have been removed and some new tests have been added:
We released Synapse 1.23.0 on Wednesday! Read all about it on the Matrix Blog. Otherwise, we'd like to highlight a few developments over the past week:
We're discussing a policy for ending support for old versions of Python and PostgreSQL. If you have opinions, please let us know on GitHub.
We've been looking at ways to improve the efficiency of state resolution, and Erik has managed to devise some algorithmic improvements that yield an order of magnitude speedup for a handful of pathologic cases. We hope to have a better idea of how this might work for real world workloads soon.
Lastly, we'd like to take this opportunity to remind you to please regularly upgrade your Synapse. Especially if you're not yet on 1.20.0, as we'll be disclosing a denial of service issue which affects older versions on Monday.
mewmew wanted to use my fork of Synapse, so I made the changes there configurable (instead of hardcoding to my user ID) and even added a readme to list the features. It might also be useful for other people who want custom room IDs and other fun stuff.
Support for receiving EDUs from matrix (typing, read indicators and presence) has been added. Protocol implementations can start using the new events, if they want to. Version 0.1.0 has been released along with this. You have to enable the de.sorunome.msc2409.push_ephemeral flag in your registration file
Hey, our team of colorful ponies proud to introduce you a new Matrix bridge to VK.com chats. It's powered by mx-puppet-bridge and so far supports almost all of core features people want to see, like replies, message edits and file attachments.
Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
Profiles should now open again on older versions of Qt.
I've been slowly rewriting the message are to be qml, to make it easier to do fancy styling. As a result emojis in the emoji-completer are now colorful and the avatars in the username completer now match the avatars used in the timeline. It also open up a few other improvements, like finally sending mentions as links instead of plain text.
We have now CI running on our self hosted gitlab instance. We will probably switch of the travis CI, once this is more battle tested. Repositories are now also automatically mirrored between gitlab and github.
Fixed a bug, where we tried to read the internal world group on your server instead of keeping that to ourself.
I released 0.0.3 a few days ago bundling a few smaller improvements like redactions, copying messages and the ability to click on more links as well as some visual cleanups. Nothing radical, but also nothing breaking (I hope).
🔗Delight (Rich vdH, Michael (t3chguy), Valere, Steve, Nique, Nad)
Improving usability
Last week
Observed user tests of people trying to use Element for the first time for personal and professional use cases
This week:
Began work on fixing several of the issues observed, like:
adding an invite people button to new rooms, so users can more easily add people;
changing copy to help people understand what DMs are
Spaces
Communities are coming back with a bang! Last week we said we renamed them to Spaces, and this week, we’ve started designing what MSC1772 would look like for users on Element, to start user testing next week.
Social login
To make authentication easier, we’ve started initial implementations of SSO in Element, exploring how homeservers & Matrix clients can support multiple SSO providers. Most of the work so far is captured in MSC2858.
The release has been blocked because a bug has been found in the end to end encryption module. It has been fixed but we want to fix damages it created on one time keys before releasing the new app version.
The new background sync service mechanism PR has been updated
We started to integrate tuist to stop to be annoyed with merge conflicts on the Xcode project file
This week:
Release!
Merge the background sync service mechanism PR and make more people test it
We’ve just merged a lot of PRs, to improve room creation form and fix some bugs.
SDK side, Dominaezzz is converting some of the Service API methods to coroutines, for a cleaner code. See for instance https://github.com/vector-im/element-android/pull/2414 . 9 out of about 45 services have been migrated so far. We have about a 45 services in the SDK (!)
This week:
Release including a new way to invite friends to Matrix and to Element.
iinuwa created issues for all of the identity service api endpoints (w/ suggested module names!) 🎉
I updated lots of enumerated types to allow unspecced values, for future compatibility & robustness
Devin R made sure a custom Content-Type in responses overwrites the default rather than creating a duplicate header. #339
gnieto fixed a bunch of bugs in ruma-federation-api
Alejandro Domínguez added support for deserializing string power levels (requires the unstable-synapse-quirks feature because these events are invalid according to the spec but Synapse used to (?) accept them)
Just released version 2.2.0 of the Matrix Ruby SDK with the help of the community, with this release support for JRuby is improved - though still not perfect, in the higher-level abstraction direct messaging rooms are now exposed for all users - as well as all that the current user has, and the lower-level abstraction sees the addition of the CS API method to get an event context.
And just to reiterate; if you're using the gem - or have questions/comments about it. please do drop into the discussion room at #ruby-matrix-sdk:kittenface.studio.
!stats command which prints you the tracked durations based on a predefined or custom range of time
CSV Export based on a defined range
!break command. Putting a break in between a record or a in/out combination.
Better handling of multiple !in commands
Improved responses
Reminders to call !out and to take breaks
Fell free to join us at #timetracking-bot:famedly.de . As soon as the repo gets opened up to public we will announce it in TWIM. Feel free to give suggestions or wishes in our room :)
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
This marks the start of Season 6 of Matrix Live (S06E01), a decision M+A apparently made on the fly just now. Incrementing a mostly-arbitrary counter... Imagine wielding that kind of raw power!
The IETF have been continuing to experiment with new chat systems, and #xmpp_bridgingmeta_jabber.ietf.org:matrix.org exists as a three-way bridge to discuss the initiative spanning Matrix, XMPP & Zulip. Really fun to see open chat converging like this :)
What type of consistency guarantees does it provide,
how many of the other homeservers you share a room with can be faulty or malicious without violating those guarantees (short answer: all of them!),
and how is it possible that a frayed-out event graph ever laces back to just a few current events,
while there is a probability larger than zero for growth to arbitrarily high limits?
While we are still in search of the right venue for peer-reviewed publication,
we went for a preprint this time to share our results with the scientific and the Matrix community in a timely manner.
In this figure, k is the number of participating servers, U_n is the current width of the event graph.
For example, with a current width of U_n = 1000 events and k = 200 servers,
we can expect that the width goes below E(U_{n+1}) < 600 events in just one step.*
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
In terms of Spec Core Team MSC focus for this week, MSC2744 has entered FCP. As such we're switching it out for MSC2844, which aims to define a global versioning system for all spec documents. Note that MSC2844 does not currently have FCP proposed. Its purpose for being in the focus this time around is for asking the team whether ideas proposed are plausible.
Thus this week's focus is MSC1544 (QR Code verification), MSC2790 (modal widgets) and MSC2844 (global versioning).
We've also started making use of a Github board to track the state of an MSC through the currently somewhat ambiguous process of putting up an MSC that is ready for review to starting FCP. You can find the board here: SCT Backlog.
Some explanation of the columns:
Awaiting SCT Input - These are MSCs that are deemed reviewable by the MSC author, and require review by a subsection of the Spec Core Team (usually one or two people) that are most familiar with the scope of Matrix that the MSC is relevant to.
Ready for FCP ticks - At least one individual from the Spec Core Team has signed off on the MSC. It is not time for others to give a cursory review - mainly sanity checking from the perspective of the part of Matrix that they work on. Note that fundamental flaws may still be found in this stage, requiring the MSC to be reworked.
In FCP - The MSC is in Final Comment Period, where everyone has a last chance to get their ideas in. This happens after most of the Spec Core Team have given their ticks from the previous column.
Done - These MSCs no longer need immediate attention by some portion of the Spec Core Team.
These definitions are all flexible as we continue to see what works for the limited resources we have available for review.
Do not be alarmed if a particular MSC is not on this board, we're placing things on here as deemed necessary, and more will certainly be added in time.
Working on the new spec platform: this week we've been simplifying the toolchain that builds and renders the spec, to make spec authoring easier and faster.
Note: We are aware of a trivially exploitable denial of service vulnerability in versions of Synapse prior to 1.20.0. Complete details will be disclosed on Monday, November 23rd. If you have not upgraded recently, please do so.
Happy RC day! The first release candidate of Synapse 1.23 is now available. For Synapse admins, we've added numerous new APIs thanks to contributions by @dklimpel, and we now support generating structured logs in a standard manner. The structured and drains logging options are now deprecated in Synapse's configuration.
Synapse 1.23.0rc1 also includes several significant bugfixes, especially around regressions in the SQLite-to-PostgreSQL database porting script, as well a fix to an issue that would prevent Synapse from recovering after losing its connection to the database. Synapse will also now reject modifications to access control lists which would otherwise result in a server blocking itself from a room. This is a particularly interesting case of unintended consequences in protocol design, and we hope to write more about it soon.
Next week should see the full release of 1.23, followed by a first implementation of MSC2403: Add "knock" feature landing in the development branch for 1.24. Look for more information in next week's edition of TWiM!
Neil is still working on P2P routing schemes and I am still working on Cerulean, but that doesn't mean nothing has happened this week! We've been busy reviewing PRs and have landed a big one this week:
Read receipts are now implemented (thanks @S7evinK!).
Usernames with = can now be created, in accordance with the spec.
Hello all. Hydrogen gained emotes (/me) this week, together with file uploads and downloads! Also, encryption keys are setup while typing instead of after sending, so you should see less momentary "the sender hasn't sent the keys yet". Also a good number of bugs were solved. Next up: image uploads!
As expected, /me commands brought a chorus of:
now that you support commands, you must add the single most important command in any decent Matrix client
Hello TWIM! From this week on we're changing how we're reporting updates on the Element clients. Each week, we'll include a section for cross-platform feature work the Element team is working on, and some sections for platform specific work too.
Spaces (previously known as Communities)
Communities are back! They've been renamed to Spaces, and on the Element team we've been busy exploring user experience and critical thinking for MSC1772, to allow users to organise rooms into useful groups.
Social Login
In an effort to reduce friction in authentication, we've been R&Ding how to allow homeservers & Matrix clients to support multiple SSO providers.. Most of the work so far is captured in MSC2858 and we're starting initial implementations in Element web next week.
VoIP: Voice & Video Calls
We've started foundational research, design and engineering to improve both the usability and stability of native matrix VoIP in Element. We've also started to look at how we can improve the continuity of the user experience between native Matrix calls and our current implementation of Jitsi.
Usability
We've been focusing on improving post-registration actions all round to make Element provide better prompts for first actions, and empty states for views on all platforms, with a multitude of fixes having landed on Element Web, iOS & Android in the last week. We've also started paid user testing to better understand specific pain points in the first time user experience, which we're looking to improve significantly in the coming weeks.
Web platform
v1.7.13 is now live! Which includes a bunch of fixes & improvements on VoIP.
Android platform
v1.0.10 is now available on the Play Store. We're also doing ongoing work to catch up with feature parity to Element Web, and Dominaezzz is migrating SDK APIs to suspend functions: no more MatrixCallback!
iOS platform
v1.1.0 is soon to be live on TestFlight for beta testers, and we'll soon be turning our attention to fixing end-to-end encryption regressions.
Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
We've (finally) released a first version of matrix_app_service.ex, an Elixir library for the application service API (hex, GitLab)
uhoreg (known Elixir fan, asked):
that's cool. Do you have any projects that are using it?
Pierre:
I don't want to "sell" it before there's some kind of poc, but with a few friends we're trying to build a bridge for ActivityPub. we're using the library of the same name by the people behind CommonsPub, of which a lot of code comes from Pleroma, so the application service appears as an ActivityPub server. it's still the beginning, and I'm just starting to think about which information a bridge needs to keep (I guess sometime I'll ask Half-Shot a big list of questions 😁)
Today might be our highest-ever Pierre-ratio in TWIM!
A new release integrates an embedded h2 database into matrix-spring-boot-bot, which makes it a lot easier to write bots with persistency. You only need a few lines of code to write simple bots (e. g. a ping bot). I also changed a lot under the hood (e. g. using type-safe Matrix-Id-classes instead of Strings).
The matrix-sms-bridge now uses the h2 database from matrix-spring-boot-bot instead of neo4j, which makes it a lot more stable and fixes some bugs. We are using it in production with hundreds of target telephone numbers without problems. I also added an optional feature to use room aliases (e. g. #sms_4901234567:homeserver) to prevent multiple rooms with one target telephone number (which is still possible).
In a few weeks you can use this bridge with an old Android-Smartphone, which acts as an sms gateway with rest api.
I've written a small blog post about a Matrix chat bot I've developed with matrix-js-bot-sdk, it was started to address Mozilla specific features first, but then I've added a few other generally useful features too! Hope you'll like it and find it useful!
Hello Matrix friends. I've been asked to share this here. I'm currently building a platform for building communities. I'm focused on deeper matrix integration with communities, private messaging built on top of matrix, and custom domain support. Feel free to check out https://gurlic.com. The matrix community is at https://gurlic.com/matrix, but I haven't had the time to post much.
We have user pages (https://gurlic.com/bob -> https://bob.monster)
The UI is a little confusing now because we have longform articles (https://gurlic.com/homer/the-iliad-book-ii) and short social media posts (https://gurlic.com/privacy/post/324848317844422661).
I pretty much want all resources to have custom domain support, and matrix rooms for all communities, and nested communities. In the near future, I hope to have matrix servers for custom domains too. And of course, the source will also be open if folks want to host.
Right now, users on gurlic get a matrix account, and communities/galleries/publications get a room too, but i'm hoping to take some time to look at much deeper integration. Dendrite and hydrogen are great to build on top off...
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Muhammad Radwan, on behalf of Tactical Tech, will present their work on The Glass Room Misinformation Edition.
What happens when we increasingly rely on social media and the web for nearly all our information? What information do we see, and what do we miss? How do we know if a picture or a tweet is genuine or truthful? And what can we do if we can’t be sure?
Popi from CHATONS will present the work done by CHATONS ("Collective of Alternative, Transparent, Open, Neutral and United Hosters")
CHATONS has been initiated by Framasoft, in the wake of the success of its campaign "De-google-ify Internet". A CHATONS member pledges to propose fully FLOSS-based online services, in an ethical and united spirit.
Hubert|Uhoreg Chathi will present on the state of encryption in Matrix, and more broadly
Why the PGP model is insufficient, what properties we want from encryption (e.g. why we want perfect forward secrecy), how the newer protocols (OTR, double-ratchet, megolm) give us some of the properties we want, and then what's coming in the future (MLS, and what that gives us)
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals
Happy November from the Synapse team! As mentioned last week, we pushed a small v1.22.1 release last Friday which fixed two regressions:
Fix a bug where an appservice may not be forwarded events for a room it was recently invited to. Broke in v1.22.0. (#8676)
Fix Object of type frozendict is not JSON serializable exceptions when using third-party event rules. Broke in v1.22.0. (#8678)
If you haven't upgraded your Synapse in a while, please do.
A major focus of Synapse is being able to meet the performance and reliability needs of massive homeservers like matrix.org. If you're curious about how Synapse's architecture has evolved over the years to meet these scaling challenges, check our our blog post from Tuesday: How we fixed Synapse's scalability!
Lastly, we anticipate releasing 1.23.0 in the next fortnight; keep your eyes peeled for release candidates and let us know if you have any feedback. For a preview of what's coming, check out GitHub for the new commits that have landed on the develop branch since our last release.
Synapse now horizontally scales across multiple python processes, as of 1.22: you can configure it so that events are no longer sent through the main proc, eliminating the single biggest bottleneck for large scale Synapse deployments. Read all about it at https://matrix.org/blog/2020/11/03/how-we-fixed-synapses-scalability
Exciting visual progress this week with actual bridging between Gitter and Matrix utilizing the virtualUser feature,
we've been iterating on the past couple weeks. Check out the image with all of the user avatars and display name goodness
to make both chats on Element and Gitter feel one in the same!
You can also check out the live demo in Matrix Live!
As mentioned in the sync on Monday, I was mostly distracted from Hydrogen this week. (sorry Bruno -BP) I did release the picture lightbox on Monday, and yesterday managed to close 4 bugs. There's also a community PR for better usability and accessibility in the login screen (keep those coming!) and after some work to make encryption more robust, I hope to do a release with all those goodies tonight.
This week, we came back to the background sync work to quickly display a notified message in the app. In parallel, we created a profiling tool at the SDK level to track performance like this one.
We are working through some low hanging fruit around post registration, blank screen interaction prompts and toast tweaks. We are also experimenting with SSO for matrix.org. Meaning that Element will give the user the option of either username/password or SSO. Finally, we are continuing our VoIP efforts and nailing down the designs, checkout Matrix Live for all the details. Next week we'll carry on with post registration UX, VoIP improvements.
Hi all! I think this is the right place to share this 🤞. I've been working on a Matrix SDK for Elixir over the last few months as a side project, it's very early days and it's currently just a bare-bones wrapper for the Client-Server API. I've also written the Elixir/Erlang bindings for Olm (currently missing group sessions) which should be added to the SDK soon. The repo can be found here: https://github.com/niklaslong/matrix-elixir-sdk and the Elixir bindings for Olm are linked in the readme.
It is the right place! Thanks cognitive_tea :D
Asked if there were big plans for use of the project
Not as yet, though a few people have reached out to me already and are building on top of it. I started it as a way to get going with Matrix dev and as a fun side project. That being said, I think providing the tools to Matrix-enable Elixir apps might lead to some interesting things.
If anyone has any precise ideas on how they would want to integrate their Elixir apps with Matrix, I'd be super happy to have a chat 👍️
Less precise ideas are also welcome, of course 🙂
Hemppa the bot is a generic bot for writing modules as easily as possible in Python. Thanks to issues with Freenode IRC bridge Hemppa got a new module for basic relaybot bridging of any Matrix rooms. Relaybots are stupid, but sometimes there's no working alternative. https://github.com/vranki/hemppa#relay-bridge
I did a talk at Arch Conf 2020 last month, on a generic introduction of Matrix and how to install a Matrix homeserver on Arch Linux. The recording has just been uploaded; it can be found on CCC's media site as well as YouTube 🙂
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
In terms of Spec Core Team MSC focus for this week, we're continuing with the widget theme: MSC2774 (widget URL template param), MSC2765 (widget avatars), and MSC2790 (modal widgets).
Updates on the new spec platform: we can render HTTP APIs (https://adoring-einstein-5ea514.netlify.app/spec/client-server/#login) and events (https://adoring-einstein-5ea514.netlify.app/spec/client-server/#room-events).
Hello everyone, I have some amazing news to share with you! While Conduit is getting better at federating, Famedly (https://famedly.com) has agreed to support me working on Conduit financially. With this news come some organizational changes:
Conduit development now happens at https://gitlab.com/famedly/conduit, please submit new issues and pull requests over there. I will update all links in the coming days.
Note: Famedly does not own the project and Conduit will stay free and open source forever!
v1.2.1 of matrix-media-repo, a third-party media repo for large homeservers, is out now. It's primarily a maintenance update though also has support for audio files if for some reason you need that.
Synapse 1.22.0 is out! We announced its first release candidate last week, and after a small rc2 the final release was published last Tuesday. We anticipate a small 1.22.1 release later today with fixes for messages not always being sent to app services (#8673) and serialization errors with third-party event rules (#8678).
We continue to see improved client join Apdex scores for matrix.org, indicating that our work in 1.22.0 to split background tasks into separate workers and allow for sharded event persisters successfully improved the user-visible performance of large homeservers.
In other news, we pushed a temporary hotfix to the matrix.org homeserver earlier this week, instructing it to drop all cross-user m.key_share_request messages. This was necessary to mitigate a bug in a third-party library which caused some clients to flood the server with requests. We'll re-enable these messages once we resolve issue #8677. In the meantime, we strongly encourage FluffyChat users to upgrade to version 0.21.1.
We're hard at work on the next release of Synapse, and the development branch already includes many bugfixes, several new admin APIs, and support for structured logging—stay tuned!
Hey folks, today I bring you a gift wrapped rainbow coated present, which could only mean Bifrost 0.2.0 is out!.
We've been making major progress trying to align bifrost with the many XMPP clients out there like Gajim and Swift, by improving it's compatibility with the various XEPs. I've also noticed a few users have started using it to bridge their Matrix and XMPP communities together which is super cool :)
It is already in fdroid, google play and ios should follow shortly. We highly encourage people to update, as it contains an important bugfix of sending out way too many key requests, which can cause bad server performance
Hot on the heels of 0.0.9.5 beta, Quaternion 0.0.9.5 beta2
is released, fixing a couple of blunders, notably inability to build with external libQuotient. Keep testing, keep translating!
We are making progress on the performance side. Now sending an event is much faster than before. We also are optimizing the crypto code. All those improvements will be available in the next release (v1.0.10), maybe next week?
Besides that, we are implementing the remaining features, we are trying to have the same level of functionality (= parity) than Element Web. We know that we have a great number of bugs to fix on the existing feature, we are also trying to fight them.
Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
This was an exciting week again. Trilene did the usual and just opened a PR, that implements the video part of call support. In my testing so far this seems to work amazingly well (ignore, that my webcam is crappy in the video, I only have so many devices...)! It's hard to overstate my satisfaction, if I am allowed to quote songs without getting a DMCA! If you want to try it out, you will need the qmlgl plugin at runtime (I had to patch some ebuilds on Gentoo to do so) and build Nheko from source. Support in our AppImage and Flatpak and for our Windows and MacOS builds will come at a later date. A big shoutout to trilene, who just works on VOIP in Nheko, without saying a word, and then drops a ready PR.
Another exciting new feature by lorendb, you can now specify --profile <profilename>, when you start Nheko, to create a separate profile. This allows you to open multiple instances and use multiple accounts at the same time (but it still uses separate instances of Nheko). This is pretty useful, if you have multiple accounts on different homeservers or are testing stuff for example. He also added a shortcut to delete the current content of the message area (Ctrl-U).
We also fixed a long standing bug, that crashed Nheko when pasting an image on mac OS, prevented copying text in some cases and build times should be about halfed again.
That's all I got today. I guess we should do a new release at some point?
I asked about trilene, who is a reliable Nheko contributor, Nico replied:
trilene seems to be a bit camera shy and prefer to work on code than take credit and talk about upcoming features. I'm surprised everytime, when a new PR is opened or trilene asks a weird question, that can only end up in an amazing contribution :3
matrix-bot-sdk v0.5.8 is out now with experimental support for EDUs being sent to appservices (per MSC2409).
To enable it you'll need Synapse 1.22.0 (released this week) and v0.5.8 of the bot-sdk. Then, add "de.sorunome.msc2409.ephemeral": true to your appservice registration file (at the root level) and turn on the de.sorunome.msc2409.ephemeral flag in your IAppserviceRegistration supplied to the bot-sdk. If all goes according to plan, you'll be able to use appservice.on("ephemeral.event", (ev) => {}) to start processing EDUs.
I hacked together a maubot-based roundtrip test that leverages the echo bot's ping command reply and reports rtt to Icinga as a passive check result. Its practical use is scientifically questionnable, but it gives a hint on end-user experience. Find it here: https://edugit.org/nik/maubot-pingcheck
Starting November 28th and 29th of this year, many bots on t2bot.io will be supporting end-to-end encryption. Though not all bots will be supporting it, this is an important milestone towards getting end-to-bridge encryption enabled on t2bot.io as a proof of being able to scale to the higher demand of encrypted rooms.
The eventual goal is to support encryption on all of t2bot.io’s bots and bridges, however we need to take small steps to get there 🙂. Note that in order to function, bots will decrypt all messages they see, but only respond to the ones they care about - this can still be uncomfortable for some rooms though, so feel free to kick them out.
For more detail on which bots are getting support and what all this entails/means, please see the dedicated blog post.
We teased this a little on Matrix Live last week (I think?), but so awesome to see this publicly announced.
Keymaker is a new WIP Project of some people (over at #serverlist:nordgedanken.dev ) that aim to provide a mastodon alike Server List and we would love to get some more input from the Community for this project on whats wanted, whats needed and whats maybe not that good to base on the mastodon counterpart.
This means we are building:
A list of Servers where Owners can add their servers
We try to do Quality controls (No fully self add. Servers get reviewed.) using a Code of Conduct Ruleset
Verified Listings using well-known files on serverside (Also allowing Admins to modify server data themself)
Server Details like:
Ping and Availability Stats (thanks to tulir for providing a API)
Public Room Lists fetched from the Server
NSFW Ratings (A NSFW tag was too generic for us)
A Section to list Rules
Admin addresses for easy ways of reaching the Server admins
Allowing to select registration state ("Open", "Invite Only", "Closed")
In a further post we plan to announce the launch of this Project as a Website. Server owners might get a ping before that to allow for setting up servers for this. This Project is not yet deployed to be used.
This is really cool. I suggested it might start to kickstart people hosting their own small, publicly open servers.
As we also allow non public servers (registration -> closed) it may also be a nice way to find communities that federate and have a look if they have a interesting room to join in the public rooms list. :)
Bubo, the community helping bot-in-progress, gets releases and a new command: breakout. The command can be used to create a breakout room from the current room. Bubo will create the room, invite and make the requester an admin, and confirm in the original room. Anyone who reacts with an emoji to the confirmation will get an invite to the breakout room. Currently breakout rooms are non-public and non-encrypted by default.
Hey all, I do engineering managery stuff at Element, if you ever wondered what on earth that actually means, here is a video of me going on and on about it. https://www.youtube.com/watch?v=2NflccKdGqU
Sell it Neil! This is an insightful chat - if you're interested in the dilemmas and thoughts of an eng manager, be sure to check this out!
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Hello everyone, normally anoa would be doing this update but today you get me (TravisR) instead. Luckily anoa has left me a script to run, so here's hoping I haven't completely messed up this week's update 😅
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
In terms of Spec Core Team MSC focus for this week, we're still focusing on the same MSCs from previous weeks to get widgets over the line. This includes MSC2774 to have widgets aware of their widget ID, MSC2765 so widgets can be pretty in clients like Element, and MSC2790 to support a form of user input with widgets.
Normally there would be a graph here of our MSC progress, however my machine refuses to accept that line graphs are a real thing today. As a replacement, here's a snowfall accumulation graph for Banff as reported by Environment Canada.
I made a simple room alias proxy: https://mau.dev/tulir/mauliasproxy. It can be used to create room aliases on custom domains without having to host an actual Matrix homeserver there. The proxy basically just responds to federation room alias queries using data from another homeserver (that data is fetched with the C-S API).
A new release candidate appears! Synapse 1.22.0rc1 was published yesterday and includes support for running background tasks in separate worker processes, as well as fixes to sharded event persisters which were first introduced in 1.21.0.
These changes significantly improved our client join Apdex score for matrix.org by making join performance both faster and less variable.
Synapse 1.22.0rc1 also includes support for a few experimental MSCs:
In the coming weeks we'll focusing on improving Synapse's resilience in smaller to medium-sized deployments, primarily through improvements to state resolution. Stay tuned!
Thanks Dan!
Today's update comes courtesy of Dan Callahan (@callahad:matrix.org), who joined Element on Monday as an engineering manager supporting the Synapse team. Dan previously worked in Developer Relations at Mozilla, and he's excited to help Matrix realize its ambitious vision for private, secure, and standards-based communication for all.
This week we released v0.2.0 on Tuesday, a reasonably big update containing various improvements over the initial beta release, and then followed it up with a bug-fix v0.2.1 release on Thursday.
Thanks to everyone who has been running Dendrite and reporting their findings, and also to contributors who have been submitting pull requests!
Changes this week include:
Dendrite no longer builds separate binaries for each polylith component, but instead has one multi-personality binary
The initial iteration of virtualUsers is in shape to merge(check out the flair 🔥) and will probably deploy in a release next week. We've split the rest of the virtualUser work into follow-up issues we can iterate on. We're working on adding room ban and spam detection support for virtualUsers to stop any bad actors. Then want to start on the actual Application Service bridge (Gitter <-> Matrix).
If you're curious about more of the details, you can track the greater GitLab epic.
I couldn't really wait to talk about before we actually hit 0.2.0 so here is a sneak peek at what's happening. We've spent a ton of time working on ironing out the bugs and making the bridge more XMPP complaint. The major headlines are:
Support Matrix -> XMPP edits
Set XMPP user displayname in the room based on their nickname (thanks uhoreg for mucking in there)
Improve performance of Matrix -> XMPP gateway messages and joining
Improve support for multiple devices for XMPP users connected to the gateway
Hydrogen gained a settings panel this week with a better session backup UX and your end-to-end device information, which should make the manual verification easier. Messages with multiple lines are also rendered as such now, which makes a big difference in usability. The app also works offline again after session backup broke that. Apart from that, several smaller fixes also landed.
Also, image decryption is well on it's way with a prototype working. 🎉
Element Android v1.0.9 is now available on the PlayStore. For the next release (1.0.10), we will optimize the performance (again!). We already have made some progress when sending a message to a room. We are now working on the crypto module and also we will probably upgrade the Realm database library, which seems more stable now. Besides that, we are still implementing the remaining features with the objective of getting a good feature parity with the other Element Matrix clients.
This week, we have been working to upgrade libs and tools to be compatible with Xcode 12.
We are making good progress to revive a kind of background sync so that a message appears quicker in the timeline when you tap on its notification. Authentication for widgets is still in progress.
Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
Nheko now shows the filename of an image on hover. (Contributed by kamath_manu)
Nheko now shows the fontname in the font selection rendered using that font. This makes it easier to know, how the font will look, once you select it. (Contributed by lorendb)
Fixed a crash when closing Nheko. While this didn't really cause issues, since you were closing it anyway, it's just bad form to crash instead of exit properly.
Removed the membercache. Nheko used to load all members on startup and store them in memory. This made startup slower. Removing it sped up the start by a nice chunk and freed about 30MB of RAM on my system. One step closer to using reasonable amounts of RAM!
Fixed excessive clipping when rendering the timeline. This prevented all batching of text messages. Now it only clips the replied to message, which makes scrolling much smoother again!
Finally, some controversial change, which is currently in master but may be reverted at some point: Nheko now automatically forwards keys to verified devices, when they request the keys, without prompting you. While that can be toggled off in the settings, it currently defaults to on. This weakens backward secrecy of e2ee a bit, but it makes it possible to recover from e2ee issues much more easily. Currently I'd argue, that it is an acceptable tradeoff. It is very hard to verify room membership of users at any point in time but the current one and room membership is not verified end to end in any way, so you need to trust the server to provide you with the correct memberlist or you just send keys to verified users. While Nheko still sends keys to all members of a room, when creating the session, it only forwards your own keys to trusted users without prompting you. Currently I think this is an acceptable tradeoff, as opening a popup with "user x wants to have session y shared in room z" is unlikely to be understood by anyone properly either. I'd be glad to hear your opinions though!
That's it for this week, but next week will be interesting too. Lorendb has been hacking on profile support, allowing you to run multiple login sessions of Nheko in separate windows and some other UI features.
In the past couple of weeks we implemented the last endpoint for the Federation
API. We are working on smoothing out some rough edges in the ruma-federation-api
crate, like a few that @Timo addressed this past week, so it will be a little while
before it's completely finished.
We've also created a milestone to track implementation of Identity Service API,
the last Matrix API we have to complete.
Finally, we've created a new Matrix room focused on Ruma development,
Quotient 0.6.2 has been released, with a couple of minor bugfixes. This release is used as a foundation of Quaternion 0.0.9.5 beta that's also getting out today - with support of (proper Matrix subset of) HTML, rich text user links (like pills, only lighter), initial Markdown support (if you build with fresh enough Qt), reactions (thanks to Karel Kosek @krkk), navigation to earlier events (thanks to Roland Pallai @rpallai) and quite a few other improvements. To make this release Quaternion had to gain its own basic HTML parser and Matrix-to-Qt-to-Matrix converter, which is likely to end up being a separate micro-library, in the hope that it will be useful for other Matrix projects building on Qt (even non-Quotient ones). A separate call to translators - quite a few strings got updated, so please head to Quaternion project at Lokalise and push the numbers at least to 80%!
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
In recent weeks the XR Finland community has been moving over from Wire to our own Matrix homeserver for encrypted secure chat. This was something that had been planned for a while but kicked off in recent weeks due to Wire suffering from serious encryption key delivery issues, causing messages for many to be unreadable in large groups. Currently we've migrated almost 300 rebels with more to come. Feedback has mostly been very positive, people generally like the Element clients 🤩 One of the interesting changes has been a huge uptick in the amount of discussion, which can be taken as a good sign. The plan is to next start bridging to some of the international XR chapters, for example those on Mattermost, Telegram and Slack. And maybe get them over to Matrix too eventually ;)
To aid in community management, we've started creating a bot called Bubo. Right now it mostly helps with maintaining rooms and allowing mass invites, but more features to help the community cooperate are coming. We were planning to utilize (actual) communities so it has some functionality for those, but decided then to wait for the communities rewrite. It doesn't yet have any releases, will update in coming weeks as features are added and releases made.
wants to clarify that XR is a decentralized movement and this does not mean other chapters will adapt Matrix - but we can hope and for sure here in Finland we'll be spreading the good experiences to other chapters ;)
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
In terms of Spec Core Team MSC focus for this week, widgets are still the main focus: MSC2774 (widget URL template param), MSC2765 (widget avatars), and MSC2790 (modal widgets).
We released our first beta last week and we've been busy taking feedback from the community and fixing problems as they have been reported to us. We will be cutting a v0.2.0 release candidate later today, which contains a significant number of important fixes and performance improvements.
We didn't publish a full changelog last week due to the beta announcement, therefore the following list includes changes from the last two weeks:
Database migrations are now ran automatically at startup
State resolution v2 performance has been improved dramatically on large state sets, seen to be running up to 20x faster in some rooms
Dendrite no longer runs state resolution over single trusted state snapshots
Monolith deployments with Kafka now work again (each component sets up connections independently to avoid duplicate consumers)
Dendrite now correctly rejects invalid UTF-8 - thanks to Lesterpig
Fully read markers are now handled - thanks to Lesterpig
The completed field is now returned properly for user-interactive auth - thanks to Lesterpig
The devices table now tracks last seen timestamps, IPs and user agents - thanks to S7evinK
A bug has been fixed in the reverse topological ordering algorithm which resulted in us giving up on inbound references after the first prev/auth event
A bug with concurrent map writes in the rate limiting in the client API has been fixed
Forward extremities and their previous events are now checked fully against the database
Typing events are now ignored if the sender domain doesn't match the origin
Duplicate redaction entries no longer result in database errors
Some bugs in /send have been fixed where the full room state wasn't requested properly before sending a new snapshot to the roomserver
The membership updaters now use database writers properly, which fixes some SQLite locking issues
The sync API no longer burns CPU processing unnecessary device key change notifications
QueryStateAfterEvents now resolves state from multiple snapshots properly
Cumulative room state is now sent to the roomserver when creating rooms locally
Missing auth events will now be retrieved from multiple servers in the room if necessary
Federation requests now have variable timeouts, allowing us to wait much longer for a remote server to process certain tasks
The /send endpoint now returns 500 errors far less often, reducing the frequency that other servers back off from sending to Dendrite
Backfill no longer uses the request context for persisting events, which was resulting in us failing to store those events sometimes
Invite stripped state from the sync API now includes the stripped invite itself, so that Element Web can display who sent the invite properly
The signing key fetch mechanism no longer gives up if it is unable to fetch specific keys
Handling of invalid display name or avatar URLs in membership events has been improved
my personal dendrite is now in roughly the same set of rooms as my personal synapse. Dendrite is idling at 180MB of RAM, Synapse is idling at 1.8GB of RAM :)
Hello everyone, this week we merged the federation branch into master. It's not ready to be used properly yet, but we're merging it as it seems stable enough for now. We also improved performance of the federation branch a lot by turning off debug logs.
Other news:
I opened two issues on element-ios which currently break register and login support on Conduit, making it completely unusable. Hopefully they can be resolved soon (https://github.com/vector-im/element-ios/issues/3736, https://github.com/vector-im/element-ios/issues/3737)
I'm working on an MSC for threading. It's still WIP, but you can take a look here: https://demo.codimd.org/s/SykbuAUwP
Thanks to everyone who supports me on Liberapay or Bitcoin!
Stonking week for Synapse as we landed sharded event persisters and deployed to matrix.org. This is the last significant component other than the main process to go through the sharding process and a major hurdle in horizontal scalability of Synapse.
Initial results look good with event persistence apdex improving, however we think there are still some significant performance improvements available through configuration and will continue to experiment.
We also moved off background processes from the main process. This is significant because it means that while the main process is not shardable it really doesn’t do anything anymore other than orchestration.
Again the initial impact looks very promising and we will continue to tune. Having moved the background processes away it also makes profiling the main process that much easier.
Aside from all of that we continue to progress room knocking put out a 1.21.2 - a bug fix release though please please ensure you are running at least Synapse >= 1.21.0 since 1.21.0 contains a XSS security fix.
Next week we will carrying on tuning matrix.org and start to look at improving state resolution performance.
Small bit of news I wanted to talk about from Bridge Island. My implementation for MSC2409 has been merged which means appservices / bridges can now listen in for incoming presence, typing and read receipt events! This means that the Slack bridge can now reliably send your typing status to Slack, and Bifrost can reliably bridge your everything to XMPP. The MSC is still in flux and could change, but for now this could really improve the native feeling of bridges :)
(Oh and I should mention anyone using matrix-appservice-bridge v2.2.0+ can use this behaviour for free)
🔗mautrix-python implements Half-Shot's new features
I've been adding support for the MSCs Half-Shot implemented in Synapse to my bridges:
Enabling end-to-bridge encryption now uses appservice login (MSC2778), which means setting up the shared secret login module is no longer required for e2be.
mautrix-python has support for receiving ephemeral events via MSC2409 in a branch, which will be merged once Synapse v1.22 is released. After it's merged, /syncing with double puppets will no longer be necessary to bridge ephemeral events.
Both of these will also be implemented in mautrix-go/whatsapp soon.
Now I just need Half-Shot to make synapse send to-device events to appservices, after which bridges won't need any hacky /syncing at all.
Several releases this week (0.1.11 to 0.1.15) with lots of changes:
url-based navigation has landed! All navigation in the app is now done through urls, meaning you can also bookmark any UI state (e.g. grid configurations).
fixed 2 memory leaks (exposed now because you can unload your session without refreshing the page)
fixed an issue with libolm running out of memory if you send a message to more than 44 devices (see issue #150).
some logical additions now we have url navigation: restoring the last url when opening the app with the default route, and a button to close your session and go back to the picker.
the app now blocks concurrent access to the same session from different tabs (it just closes the session in the non-active tab). This will prevent multiple syncs tripping over each other writing to indexeddb (e.g. ConstraintErrors and friends).
updates are announced in the app (for now through a confirm dialog, but will use an in app notification once we have it)
fixes updates not installing on iOS, by having an update prompt. To get this update on iOS though, you'll need to unpin the app, and pinning it again. You'll need to login again after this. All future updates should be installable through the update prompt once you have 0.1.15 though, you won't have to do this again normally.
We are currently preparing the release of the version 1.0.9 of Element Android, which contain searching messages in clear rooms and a lots of other improvements and new features.
The SDK 1.0.9 will also be released, with an updated readme, and a brand new sample app, written by Ganfra. It will help developers to start using the new SDK and can be found here: https://github.com/matrix-org/matrix-android-sdk2-sample. This sample app is able to let the user connects to an existing account on any homeserver with password login, display the room list, display a room timeline and send message to a room.
a brand new sample app
I published the first preview of my Sailfish client called Konheko. While you can run Android applications on Sailfish, they usually are a subpar experience, since they really don't fit the platforms design and style and also usually don't properly send notifications.
So about a year ago I started working on a Matrix client for SailfishOS, but I never really made much progress. Well, last weekend I did, and so it can now send plain text messages as well as various forms of media messages, I made a basic application icon and I've been using it this week already (for unencrypted rooms).
It is still missing a lot of features, but if you want you can install it from OpenRepos. Sources are available here. Just be aware, that it currently stores all messages in RAM, so every restart will take forever to load your rooms and it may run out of RAM at some point. Storing messages in some database will come at some point. Also, a lot of menus may lead nowhere, since those are just placeholders for me atm.
Recently, I tweaked Element-web to feature a few changes similar to SchildiChat for Android.
For now, it's probably best seen rather as a proof-of-concept than a finished product, as there are still some layout bugs, and no settings available for the added features (I know some people prefer separate lists for direct and group chats).
I consider it usable though.
Particular changes compared to upstream Element-web include:
A common section for groups and direct chats in the overview
Message bubbles
Bigger items in the room overview
A different dark theme, similar to SchildiChat for Android
I don't know how much I will work on this in the future, but I figured it might be interesting to share either way. Maybe even someone with more web-development skills than me might want to help improving it :)
The current version of SchildiChat-Desktop is available for Desktop here, and I host the web variant here.
If you want to build it yourself, check out this repo.
I wrote a Wake-on-LAN bot to wake up hosts by sending a matrix message. It is configurable with multiple hosts and has a list of users per host who are allowed to wake it up. It's using the matrix-rust-sdk, source is available over at https://git.jcg.re/jcgruenhage/matrix-wol, and if anyone has questions, feel free to join #matrix-wol:jcg.re.
When I asked what this is used for:
I have stuff on my workstation that I need access to most of the time, but keeping it running uses too much power (but I did it anyway so far), this is so that I can suspend it when I leave but can still power it on when I need something from there on the got
I was invited to the German Podcast MacMittwoch (no, it's not only about Macs) to talk about Bridges and Bots in Matrix. It was a very interesting and funny round.
This is a script I've created for use with MSC2545. It allows easy uploading of emoji packs to Matrix rooms. Feel free to check it out on Gitea, or join the project room #matrix-emoji-upload:blob.cat if you have any questions/comments/issues.
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals
In terms of Spec Core Team MSC focus for this week, we're going to be keeping the same three MSCs as last week, as their review is still a work in progress. As a reminder, those are MSC2774 (widget URL template parameters), MSC2765 (widget avatars), and MSC2790 (modal widgets).
It aims to be an efficient, reliable and scalable alternative to Synapse.
We encourage early adopters to try it out starting with v0.1.0 in Monolith/Postgres mode.
Dendrite supports a large amount of the Matrix spec including:
All room versions
E2E encryption (but not cross-signing)
Federation
and many others!
We'll be hard at work in the coming weeks ironing out any issues that crop up as Dendrite enters the global
federation of Matrix servers. At present, Dendrite has not been optimised for memory/CPU usage, nor does it
fully support sharding, so there is still much scope to improve the already impressive resource usage whilst
This week I worked on a new feature: the Admin Room. This is a room that is automatically created for the first user of a new homeserver. This room will be used to send warnings to server admins in the future and can be used to control Conduit (like querying information, running cleanup, banning servers, hiding public rooms or restarting Conduit). Power levels can be used to control who in the room has access to which commands. The admin room will work over federation, so it's easy to help someone fix his server.
Other news:
Federation is now disabled by default and can be enabled in the config
Bug with typing notifications is fixed
Thanks to everyone who supports me on Liberapay or Bitcoin!
This week we put out v1.21.0rc3, which fixes a duplicate messages bug. We expect the full release to go out early next week.
After a long and sometimes painful journey we plan to run sharded event persisters on matrix.org from Monday. All being well we will start running background processes on their own worker to free up the main process.
These two projects combined should noticeably improve performance of matrix.org and any large scale deployments.
Aside from that we are continuing to work on room knocking as a feature and have been considering ways to reduce load attributable to state resolution.
The native Gitter <-> Matrix bridge is underway and we're working on virtual users which is the first piece to making the bridge feel native. You can see what this will look like in this ominous and exciting tweet: https://twitter.com/gitchat/status/1314252991705292800
There is also a GitLab epic you can track which breaks down some of the tasks and holds more details of what we're thinking of for the bridge.
Asgardians rejoice, for Bifrost development resumed this week! We've worked extremely hard this week to improve general gateway performance, edge case bugs and basically plowing on with the project. A 0.2.0 release will shortly be due with the new changes, and matrix.org is already running the latest and greatest. You can checkout the project over at https://github.com/matrix-org/matrix-bifrost
Former Fractal GSoC intern Alejandro Dominguez 😎️ got his Integrate matrix-sdk (I) merge request in “Ready for review” state. That’s a +3910/-5439, 38 commits diff‼️ poljar, who is the main contributor of matrix-rust-sdk, already had a look and provided some insights, but given 😱️ the sheer size of the changeset, the moar 👀️ we can get on it the better.
FluffyChat now runs on native Linux desktop (x86).
You can download Linux tarballs from the CI and run them on any system which has sqlite3 installed (which all distros may have). If you have libolm3 installed, you can also use e2ee.
Oh and by the way. Its NOT electron or any html5 stuff! Its a native Linux app.
We are working on packaging solutions like snap and Flatpak. 😊
Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
This week we finally cleaned up the remaining parts of Chethans GSoC work and merged it. This means Nheko now has partial support for cross-signing. You can either start the verification from another client or when viewing the profile of a user. There are a few things still missing though:
Nheko does not use the verification status at all at the moment. So while you can verify your master key and let others verify your master key, from Nhekos perspective there is not much benefit yet. You will however show up with a green checkmark in other clients.
While nheko supports verifying your own master key and after that will also let others verify your master key, it does not yet download the self and user signing keys. This means you won't be able to upload cross-signing signatures for your own devices and other users master keys yet. I'll see if I can implement in the next weeks or so (since literally only the private key download or sharing is missing).
Nheko can't bootstrap cross-signing yet. This means you need to bootstrap it from Element or another client once. That will probably take a bit until this is complete.
The UX is still WIP. Currently it shows a lot of unnecessary buttons, that are a bit confusing as well as not enough green checkmarks, so that needs some work.
Other things that happened in the mean time or are work in progress:
Decryption errors should be reduced by a bit now. This is still work in progress though. With device trust landed now, we should be able to do automatic key requests, sharing and key backup soon.
Lurkki is currently cleaning up the design of the built in video player.
manu_kamath is improving the design when showing images at the same time.
LorenDB has been updating our screenshots and working on an Esperanto translation.
trilene has been ever busy improving edge cases around the Voip support in Nheko.
So far Hacktoberfest seems to be going well with quite a few people trying to clean up some of the rougher edges in Nheko. If you are using Nheko and think some things could be better, why not try to fix it yourself? October is the perfect time for that!
Featurework of this cycle is also slowly nearing its end. Once cross-signing, voip and the new event store are stable, we plan to improve the UX a bit more as well as some other annoyances and maybe we can do a new release then! A nice and spooky Hacktober everyone!
Fixed a race during cross-signing key upload at registration time
Clarified when you have unsaved changes in profile settings
Finally, the improved widget support is looking very exciting. Here is a design grab to give you an idea of what to expect. Note this is not a real Element screen shot, the base functionality is there, but the look and feel of the final version is subject to change.
This week, Hydrogen gained better caching (and a little over-eager on Safari), room list filtering, a grid layout to show up to 6 rooms simultaneously (see screenshot) and work is well on its way for url-based navigation within the app.
also adding the screenshot on iOS that I didn't get in last week, showing the full range of display formats supported 🙂
The app also received some visual polish, thanks to Nad, our designer.
This week, I have improved the scoring rubric, so that the verdict given to you by the tester more accurately reflects the likelihood of connection establishment.
I have also added more hints on what the verdicts mean.
Finally, I have added workarounds to support Chromium browsers that decided to break their WebRTC API so that they could pedantically rename ip to address. In any case, it now seems to be on par with Firefox.
The VoIP tester is now roughly 'usable' but please note that IPv6 support is totally untested (I don't have IPv6) — it is also suspected that some browsers won't try IPv6 anyway.
There are also some more features on the wish list, such as screenshot safety and extra help when things go wrong.
Versions 3.2.0 and 3.2.1 of libolm have been released. 3.2.0 supports fallback keys (MSC2732) in the C library and the JavaScript binding, and includes several JavaScript/TypeScript fixes and improvements. 3.2.1 was just a fix to the TypeScript definition file.
This week, q-b implemented moderation policy events, bringing us to full compatibility with r0.6.1 of the client-server API. iinuwa added the only endpoint we were missing from the appservice API. Additionally, I improved Ruma's JSON canonicalization, making it both simpler and more efficient.
Due to twoissues regarding future-compatibility, we're still a while away from our next set of releases, but we're working on it!
ObjMatrix (GitHub) is a new Matrix client library written in modern Objective-C for ObjFW. Because it uses ObjFW, it is not limited to Apple platforms, and instead is extremely portable and should run on basically everything.
This is currently in its very early stages and is intended to be suitable to develop bots and clients with very soon. Being suitable to develop bridges with it is another goal for later.
Since ObjFW also works on platforms that are usually hard to support or port software to (e.g. MorphOS), this will hopefully also bring clients for these less mainstream operating systems. I'm especially hopeful about MorphOS here, since ObjFW already runs quite well there (parts of it are included with the OS even!), and hardware on which MorphOS runs on is usually not powerful enough to run Element Web. And eventually, I will want to have a Matrix client on my Amiga, too 😉 (which this should enable).
This currently does not have a decicated Matrix room yet, so all discussions about it currently happen in #objfw:nil.im. So please feel free to join there and follow along development. And contributions are of course very welcome!
We’ve put together a write up on how to deploy a Noteworthy Hub (transparent TLS proxy) in order to make it easy to run a federation capable matrix home server behind a firewall or NAT: https://noteworthy.tech/hub
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
It's true! Element has acquired Gitter from GitLab and will be implementing
Gitter's current chat features in Matrix, before rebuilding Gitter as a Matrix
client! Before all that however, the first step is to build a proper, modern
bridge between the two networks, replacing the old one that's been around for
years.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
In terms of Spec Core Team MSC focus for this week, after dropping off MSC2414 in the FCP bucket, we're heading back for another big swing at widgets. MSC2774 (widget ID URL parameter), MSC2765 (widget avatars) and MSC2790 (modal widgets) are the focus for this week 🙂
being totally shook up about the Gitter announcement for a good part of the week, I completed the work on MSC2312 (it's about Matrix URIs, ICYMI) and it hopefully won't be long before it becomes the actual standard to share Matrix coordinates in popular web browsers. Ok, who am I kidding - before Matrix clients get on with adoption.
This will allow people to post links to Matrix rooms/messages/users and when clicked will open right in your favourite Matrix client. Super convenient and great for adoption!
Build 38 of the iOS P2P Demo, as built using Element iOS and Dendrite, has been submitted to TestFlight and will hopefully be available for testers shortly (pending Apple approval)! It features lots of updates in the Dendrite backend which should hopefully make it more reliable.
This week we put out a new release candidate -1.21.0rc2
Highlights include
Add experimental support for sharding event persister. (#8294, #8387, #8396, #8419)
Add experimental prometheus metric to track numbers of "large" rooms for state resolutiom. (#8425)
Add prometheus metrics to track federation delays. (#8430)
Fix messages not being sent over federation until an event is sent into the same room. (#8230, #8247, #8258, #8272, #8322)
Fix a regression in v1.21.0rc1 which broke thumbnails of remote media. #8438
Aside from that we are working on moving background processes away from the main process, actually getting the event persister sharding onto matrix.org and trying to improve Synapse stability generally.
Dendrite is nearing beta! Today we will be cutting a candidate 0.1.0rc1 version after a week of hunting and fixing bugs. We are on track to release version 0.1.0 next week, at which point we will be inviting people to try installing and using Dendrite!
Changes this week include:
Initial sync is fixed after a bug caused us to fall back on incremental sync
The Content-Type HTTP header is now handled properly when MIME-formatted
Internal API calls over HTTP in polylith mode now use their own HTTP client with higher timeouts
Dendrite now tries harder to find missing auth events, using fetcher workers
Dendrite no longer falls back on /state unnecessarily, which was the cause of a major memory leak and high CPU usage
Federation HTTP calls now include the User-Agent header
The event depth field is now ignored in the federation API
We now report the password change capability properly (thanks bn4t!)
Registration flows now include the completed field after a failure (thanks Lesterpig!)
A bug in the previous event updater in the roomserver has been fixed
A bug where we didn't check our own old verify keys when verifying event signatures is fixed
A bug where we didn't handle event ID domains being different to the event origin in earlier room versions has been fixed
If contributing to Dendrite sounds like something you would be interested in, please take a look at these issues and join us in #dendrite-dev:matrix.org! There's also #dendrite:matrix.org for general Dendrite chat and updates and #dendrite-alerts:matrix.org for release notifications and important alerts.
This week in Matrix Construct supports aarch64 (ARMv8) architectures. The experience has been phenomenal, with performance exceeding all expectations. Last week I wrote about all new vector code inside Construct using SSE through AVX-512 hardware acceleration in servers; that same code is now accelerated by SVE (Scalable Vector Extensions) on ARM architectures when you compile with clang.
This result is important: ARM virtual machines are offered at a significantly lower price compared to x86 from the same vendor. The systems are cheaper, require less power, but generally perform worse. The trick is to optimize the software for the weaker hardware. The benefit allows, for example, Matrix server hosts running Construct to make th eir profit margin from the lower TCO, getting more out of every computing cycle.
Currently it supports bridging messages, reactions and signal read receipts. End-to-bridge encryption also exists (mautrix-python does all the work there). The bridge can be linked as a secondary device and possibly even registered as the main device, but I didn't actually test registering yet. Setup instructions are currently somewhat non-existent, but it's mostly the same as my other bridges plus signald as a separate daemon.
mx-puppet-discord got updated to the newest discord.js version, meaning you have to update if you want to continue to operate it, due to discord having changed their gateway url!
mx-puppet-discord now also supports the intent stuff, so be sure to update by 7th oct
If you run mx-puppet-discord (like I do), make sure to update by October 7th or it will stop working!
Hey folks, I wanted to give another shoutout to say that we are still accepting PRs as part of hacktoberfest. Contributing 4 PRs will get you a T-Shirt (sadly not a Matrix one). Obviously, please ensure your PRs are meaningful (no copyright adjustments, typo fixes).
In the hope of expanding the number of people contributing to the matrix-appservice-slack repo I have spent a chunk of my morning improving the issue descriptions and labelling up issues. If you are interested in fixing a little annoyance with the slack bridge or just fancy writing some typescript see the good first issue label on the repo.
The matrix.org team are delighted to bring you the latest in Slack bridging technology. Do not let the
minor version bump fool you, this release is packed with the good stuff. The headline feature is that
our phase 1 encryption feature has landed and is free for users to experiment with. Head over to the
More browser compatibility work this week, making Hydrogen run IE11 on Windows 7, and on Safari on macOS and iOS (still with some caveats). Also fixed several bugs:
fix for unable to open session after a synapse bug manifested itself
prevent the app locking up when you start the app with previously unsent messages
fix sync errors being reported as "null" in the banner
handle timeout during initial sync (important for large accounts) (although I have seen 1 report that this still isn't fixed, please report if you can't login with a large account)
Element Android: Version 1.0.8 is now available on the stores, it fixes issues with verification and PIN code among other issues (see https://github.com/vector-im/element-android/releases/tag/v1.0.8 for more details). Now we are working on improving performance when sending messages to rooms, and also improving global UX, especially of the home (rooms list). Search messages (in clear rooms for the moment) is coming soon, and it will be also possible to filter the room members list.
We will also spend some time on the new Android SDK, https://github.com/matrix-org/matrix-android-sdk2, which is for the moment a quick extract of what we have in Element Android. We have to take care of it as a real product now: document it properly, set up CI, export Javadoc, develop a sample app, etc.
Element for Nextcloud v0.6.11 has been released this week. The new version comes with various bug fixes, dependency upgrades, and an upgrade to Element Web v1.7.8. The version is also compatible with Nextcloud 20 which is being released soon.
Over the past couple of weeks, we've received PRs for all of the remaining federation endpoints, and all but one have already been merged!
Now that the end is in sight, we're turning our focus elsewhere. We're working on cleaning up and fixing a few bugs in our event signing code and soon will create tracking issues for filling out the Identity Service API.
All of these users are Telegram/Discord users that have been brought into Matrix over the last 30 days. This doesn't appear to be a temporary spike either: over the last 8 weeks t2bot.io has been hovering at 900-950 thousand monthly active users, up from 600-700 thousand. Record-setting traffic levels have also been achieved, with matrix.org being able to keep up for the first time in a long while.
Overall it's a good sign to see so many communities making the jump to Matrix and sticking around ❤️
I added zabbix-bot to my zabbix-matrix repo, which can get information about current problems from zabbix-server and send it to matrix user. https://github.com/progserega/matrix_zabbix
I've found that having your systems reporting in a room while you chat around
it can be really productive. Props to supporting yet another monitoring platform!
While 1-1 calls benefit from end-to-end encryption due to WebRTC, Jitsi group calls have always only benefitted from transport encryption.
A while ago Jitsi announced that they were adding E2EE to Jitsi. But did you know that it's using Matrix's Olm encryption under the hood? It's currently available as an experimental feature on https://meet.jit.si!
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Rank
Hostname
Median MS
1
fairydust.space
368
2
blob.cat
543
3
nuclearlimes.co.uk
580
4
nuclearlemons.uk
889
5
conduit.rs
912.5
6
mailstation.de
1075
7
matrix.org
1092
8
test.zemos.net
1123
9
shortestpath.dev
1251.5
10
chatcloud.net
1338.5
We also now have a room where the ping bots are only hosted on non-Synapse
servers! See the scoreboard below.
Hello all, and welcome to this week's addition of This Week in Matrix!
Matrix is an open network protocol for secure,
decentralized communication on the web.
My name is Andrew (aka anoa), and I'm a Synapse developer at Element.
Thanks to Ben for letting me take over the reins of TWIM for this week! I'll
actually be doing the same for next week as well, so please adjust your
clocks to Anoa Nonstandard Time accordingly.
It's demos week again. This time around we've got the following lineup:
Michael shows off all the new widget goodies coming to Element Web!
Bruno shows off Hydrogen's new encrypted session backup support!
Ismail details the new room creation flow on Element iOS!
Jorik presents his work on revamping the UX of https://matrix.to that he completed for his second summer internship at Element!
Hubert fixes another class of failure-to-decrypt messages edge case with Device Dehydration!
Half-Shot closes off with the tale of Encrypted Bridges!
Don't forget that Matrix Live is also available in podcast form, if you're into that sort of thing. Search for "Matrix Live" wherever you get your podcasts.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
In terms of Spec Core Team MSC focus for this week, we've been rather busy with implementation, so we'll be continuing on with the same focus as last week. As a reminder, that's MSC2414 (making reason and score optional on reports).
This week has mostly been spent trying to improve stability and to fix bugs ahead of the beta release, which is so far still planned to go ahead in the next two weeks.
Changes this week include:
Room version 6 is now the default for newly created rooms
Soft-fail of events that aren't allowed by the current room state is now implemented
Support for configuring old_verify_keys has been added to the Dendrite config
Correct formatting for signing key IDs is now enforced in the configuration
Initial support for peeking over federation (MSC2444) is in progress and it "even works!" (thanks Matthew!)
Federated joins will now continue being processed even if the client gives up on the join due to a HTTP timeout
Backoff code has been refactored a bit more, and now correctly affects device list syncing
/make_join now errors correctly if a federated user tries to join a room which all members have left
A bug where a single user could start multiple simultaneous federated joins to the same room has been fixed
Some initial (but unfinished) support for the /key/v2/query notary endpoint has been added
Signature verification has been updated to not fail if the event origin field is missing (although it still requires a signature from the domain of the sender field)
A number of places where we use SQL transactions have been updated with safe wrappers (thanks samcday!)
A couple of error codes on invite endpoints for room version 6 JSON violations have been fixed
Spec compliance has improved slightly for federation:
Client-Server APIs: 56%, same as last week
Server-Server APIs: 77%, up from 74% last week
As always, if contributing to Dendrite sounds like something you would be interested in, please feel free to join us in #dendrite-dev:matrix.org ! There's also #dendrite:matrix.org
This week we released 1.20.0 (and 1.20.1) highlights include shadow banning support and including unread message counts in the sync response. This will help client developers and is a precursor to improving notification support.
We’ve also been looking at adding monitoring to get a better sense of which rooms are most expensive from a state resolution perspective, we also want a better way to track federation lag.
Up next we’ll move all background tasks away from the main process and try it out on matrix.org, we are hoping for a 10-15% saving in CPU. Event persistence sharding, specifically the new stream token format is on hold slightly while we work on a nasty race condition on start up of the event persister. We hope to get back to the main sharding project next week.
This week in Matrix concludes a summer of Construct where several transformative rounds of optimization took place. I'd like to talk about these achievements and why they are important for
future directions.
Over the summer, Construct introduced new vector extensions to the project. As server software, our primary target is the datacenter which (for now) is dominated by x86 hardware. The latest features of x86 chips include AVX-2, AVX-512, and on-die GPU. These advancements are important because they help mitigate current limitations of hardware, such as the cubic relationship between a processor's frequency and its power consumption. These limitations mean that CPU cores aren't completing more cycles-per-second every iteration of their development -- they're not getting much faster.
The problem with threads is that they can introduce a lot of complications to a project. Construct is able to forego multi-threading as a network server because it is primarily IO-bound. The benefits to a single-thread design in both performance and simplicity cannot be overstated. This is why Construct stands to benefit the most from features which allow more work to be accomplished at every individual computing cycle.
Construct undertook several endeavors over the summer which directly leverage platforms featuring SSE, AVX, AVX-512, etc. As an added bonus, our approach is designed to effortlessly port to ARM's Scalable Vector Extensions (SVE) as it becomes available (inside datacenters too!). Our focus has been JSON, Unicode, and finally Base64. Detailed explanations for each of these will need to be discussed in their own posts, but in summary:
Matrix specifies a canonical form of JSON which is not necessarily the same as the JSON the server receives. Therefor it is imperative that Construct is liberal with what JSON it accepts and correct with its transformation into canonical JSON. Over the summer, a portion of Construct's canonical transform received some optimization to go beyond the default method of character-by-character read-modify-write. As seen in [1], a streaming hardware-accelerated transform now processes at least 16 characters at a time (with more possible) without forward branching except for the parts where transformation needs to occur. One of these transformations is UTF-16 to UTF-8 surrogate conversion, which leads into the second endeavor.
Construct introduced a completely branch-free Unicode toolset in [2]. Among the functions offered is a custom transform of UTF-16 surrogate pairs to UTF-8 sequences. Using the new vector registers, Construct can transform two UTF-16 surrogates in parallel to output two UTF-8 sequences, or two UTF-16 surrogates in a pair to output one UTF-8 sequence. Unfortunately, these specific functions were written at fixed vector widths, so more work needs to be done to really take advantage of the widest hardware. Each surrogate is 6 bytes, and a surrogate pair is 12 bytes; therefor we cannot make use of the last 4 bytes of a 16 byte vector. However, with a little more work this approach can be extended to a 64 byte vector, capable of decoding 5 surrogate pairs and 10 individual surrogates in parallel!
Professor Daniel Lemire recently published a paper about fast Base64 from hardware acceleration in [3]. This approach is extremely elegant on the 64-byte-wide AVX-512 system. Prior to Construct's implementation of this in [4], the Boost library base64 encoding and decoding took roughly 20 and 25 cycles per character respectively. Our implementation on the same system, an old system with only SSE2 (not even AVX-512!) yields 5 and 6 cycles per character!
All of this helps lay a foundation for Construct to introduce Federated Media Rooms sometime in the future. Currently, Construct stores media in a separate database. Recently there's been work on a separate branch at [5] which stores actual file block inside events using Base64. It is for this reason sub-cycle and branchless JSON parsing and Base64 encoding is essential for maximum performance. The result is worthwhile, as the latency for querying the media database is slower than parsing and decoding the event content already in-hand.
It's really exciting to see Homeserver development ramping up from all
angles, and nice that the protocol warts are slowly getting ironed out in the
process.
Thanks to Arkaniad, v2.7.0 of my Matrix Helm chart for Kubernetes is released with support for exporting Prometheus metrics. It pairs well with the Synapse Grafana dashboard.
The synapse docker image from AVENTER (https://www.aventer.biz), does support PowerPC (ppc64le) and ARM64 architecture now. But at the moment only under the docker tag "ppc". https://hub.docker.com/r/avhost/docker-matrix/tags?page=1&name=ppc We will be happy to get feedback.
As a Synapse developer, it's great to see the community making personal and
enterprise Matrix deployments more accessible!
Timeline randomly resorting while more history is being fetched
Automatically request history if the "load more" button is on the screen
Sorunome briefly mentioned afterwards that there is no 0.19.0 due to some accidental messed up tagging, and that it was easiest to just call the new version 0.19.1.
Released 0.1.0 (and 0.1.1) this week with read-only session backup enabled 🎉 Also doing more work to make Hydrogen work on IE11 on Windows 7 (it does already for Windows 10), Safari and other browsers where you get TransactionInactiveError during login, hope to release that soon.
I need to give Hydrogren a shot myself, the quick speeds and low RAM usage
are really attractive.
This week, room settings have been updated with a new intermediate screen. The codebase saw the introduction of an AppCoordinator (in swift) which will help us to have a better control on navigation within the app AND to use swift from end to end.
Polyjuice Client v0.3.0 has been released. This release includes many breaking changes. ("Breaking" in the sense of API changes, rather than the sync process suddenly failing to work, which was already featured in a previous release.) This release also includes many changes, such as the client managing more bookkeeping, detecting if it got logged out, and supporting more Matrix features. See the release notes for more information.
I present you a new project I've been working on for some time. It's a bot that allows you to use the admin API through matrix, by typing commands to the bot. The inspiration comes from the OperServ-like bots that allow IRC operators to administrate an IRC server.
The bot exposes some of the available admin APIs and aims to provide some more high level commands by combining different APIs. There is currently one "high level command", !room garbage-collect, which allow you to purge from your HS all rooms without local users.
The project is written in crystal and is my first project in this language. It should be stable in normal conditions, but don't hesitate to fill issues. A Docker image is provided for more convenience.
benpa wanted a bot that replies with a songwhip.com link whenever someone sends a music link (youtube, spotify, apple music, etc), so I wrote a small maubot plugin to do that: https://github.com/maubot/songwhip
It's available at @songwhip:maunium.net
I'm also desperately in need of this for the 10 open.spotify.com links that get thrown at me every week. Thanks Tulir!
It supports E2EE and can be easily extended with Python plugins. Can be used e.g. for GitLab notifications, automated user invites, room creation, and of course can be programmed to react to any message posted in a room.
Soru made a new thing, matrix-gotify. It is a gotify plugin to receive matrix notifications. This means, you can now receive matrix push notifications on your android phone self-hosted without the need of google services! That is why putting the full event in the push notification is actually not a privacy leak in this case.
This plugin could also be used to receive push notifications on any other kind of device that gotify supports.
Please note that this is independent of any matrix client - you don't even need to run one to be able to use this!
I asked whether an OpenPush-like solution be
built on top of this , and Sorunome responded that someone had already
started working on just that! https://github.com/gotify/android/pull/115
This would allow other matrix clients on your phone to get their push
notifications through your gotify client, instead of needing to run a process
each. Great for battery life without proprietary Google blobs!
js got Synapse running in a car on a German highway:
the setup is a cigarette lighter to 2x 230V converter, one being used to power the RockPro64, the other being used to power my notebook. My notebook is connected to the hotspot from my phone, while also being connected to a USB ethernet, the other end of which is plugged into the RockPro64.
The notebook then acts as a gateway, as well as SSHing to a tiny VPS with -R, to get a public port, and forwarding that traffic to the RockPro64.
Maybe one day we'll all have an embedded Synapse homeserver in our cars. P2P E2EE
car comms anyone?
I have been through the perils of setting up a TURN server and having VoIP continue to fail, whilst spending hours scratching my head and staring at Wireshark without getting anywhere. When I was given free reign over project choice last year during my internship, I chose to start a tool that tests your homeserver's VoIP configuration, hoping it would be useful to both me and the community at large.
It saw some progress but I never managed to iron out some of the 'last few things' or put a pretty front on it — until about 2 weeks ago, when I got some more time to do it.
There are known issues — such as the scores being very arbitrary, wrong and misleading; it crashes Chromium every time (on my machine) and it completely misbehaves in Brave Browser (on my machine). It may also not be the prettiest (but hopefully it is at least somewhat inoffensive). I hope to work on these annoying issues soon.
I have deployed a test instance at https://test.voip.librepush.net — it can be tried in a WebRTC-supporting browser (probably Firefox if you want half a chance). Please don't take it to heart if you get a Fail or Poor score — it's probably not your fault. :)
Oliver was interning with us at Element this Summer. Fixing up and releasing
this VoIP tester was part of it, but he's still working on it in his spare
time even though his internship has finished :)
Through this I found out that I forgot to turn on my turnserver on my
homeserver again. Thanks Oliver!
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
I changed the formatting of the plot a bit to make it more readable. Note that it is now using a log scale which allows us to see more data at the same time.
Now that we have multiple somewhat federating Matrix homeserver implementations, I decided to make a ping room where all the echo bots are hosted on second-gen homeservers: #ping-no-synapse:maunium.net. Synapse users are still allowed to join and !ping, but all the pongs will come from non-Synapse servers.
Based on observations in that room so far, the new server implementations don't like eachother very much.
Category Atom Feed
