farooqkz

Farooq's KaiOS related experience with his KaiOS phone

#infernojs #inferno #matrix #matrixclient #voip #voicecall

It has been a few days which I have started the development of the app. Just like the prototype I had coded before, InfernoJS is used as framework to create the app.

You can see the Github repository here.

Why InfernoJS?

InfernoJS is a fork of ReactJS but with a performance comparable to Vanilla JS which is Javascript without any framework. You can see benchmarks here. Let me explain the reasons Inferno has been chosen. If you are already familiar with React-like frameworks, you can skip this section and jump to Why Github?

Just like React, an Inferno app is made of components and each component is parent of some other component or components. Of course there are “low level” components which are home to real CSS and HTML. But you code those low level ones once and them everywhere without a need to waste your time to think about HTML or CSS again. As an example, for Login component this is the tree:

Login
  - Header
  - ListView
     - TextInput(for homeserver address)
     - TextInput(for username)
  - SoftKey

As you see when writing Login component, I only need to know about usage of Header, ListView, TextInput and SoftKey components. I don't need to know about their details and I can focus on important things. Regarding these child components I only need to write them once and use them everywhere which makes more free space in my mind to care about stuff I should really care about like the logic of my app and other things.

This makes my code highly readable and components which I write are reusable. I could also distribute the components which are common so that someone else or I, would only need to use them afterwards. Just like how you use a library or module without caring about their details.

You can check code for Login here

Why Github?

If you were already following me and my projects, you could see that I was always using notabug.org for projects but for this app I've switched to Github a service by Microsoft running a propriety software. But I have reasons to choose it over NotABug and Gitlab as well which I'll explain them in two parts.

The main reason I've chosen Github over NotABug is to use Github Actions to improve the development of the project and automate stuff. Of course I could combine NotABug with an external CI/CD provider but I haven't got any machine to run CI/CD jobs on it and also I wanted to focus on the project rather than other stuff.

However I could use Gitlab which provides a similar service to Github Actions(actually Gitlab had this before Github as far as I remember) for free of charge but I live in Iran in which because of sanctions I can't use Gitlab even for public free/open source software projects directly. An alternative way would be using some kind of VPN or proxy to change my IP address to one which is not from Iran but that reduces the quality of my internet connection which being in Iran is already very low. To realize what do I mean by “very low”, I should tell you that my wired home connection is 2Mbps ADSL which in reality is a bit lower than 2Mbps...

Nokia 800 Tough image

#matrix #voip #voicecall

As of now, enough money has been raised for the project to get a Nokia 800 Tough but not enough to even start the first stage of the project. Thanks to all of you who donated and also those who wanted to donate but they couldn't due to any reason(including an empty pocket!).

If you wanna still donate, you can send some Bitcoin Cash to the address I shall write in the project's page but please DO NOT send anything to the addresses specified in the previous post.

Because of what was not raised, I don't promise working 15 hours a week on the project but due to what was raised I cannot leave the project so I plan to work on the project about 7 hours a week starting from 1-2 weeks later.

In the next post I will write about status of the project and also address of project's page on Github.

Matrix logo

As of 2021-06-19 72 USD out of 200 USD has been reached.

#matrix #voip #voicecall

The story

It had been a very long time which I wanted to start developing a Matrix client with voice call support for KaiOS. Before that I had tried developing a Jitsi Meet app for KaiOS letting KaiOS users have group voice chat with their feature smartphones, which failed due to their ToS.

What is Matrix?

Matrix is an open and federated protocol for chatting which supports End to End Encryption, 1 to 1 voice and video calls(support for group calls is experimental). Is it also possible to integrate other chat applications like XMPP, Telegram, Discord, IRC, etc into Matrix using bridges.

What am I going to do?

I want to create a Matrix call only app for KaiOS letting users of this platform to use 1 to 1 calls, especially voice call. I need at least 350 US dollars to work on it as a job about 15 hours a week and to get a Nokia 800 Tough which costs me about 120 dollars 90 dollars in my country.

Note that I've got a 8110(aka the bananaphone) but its keypad is almost dead.

Donators' name as well as the amount they've donated shall be written in Credits/About tab of the app visible to everyone who uses the app. If you want your name to be written you MUST email me your crypto public key before making the transaction

hapu [at] riseup [dot] net

I already have written a prototype of a Matrix app which let's you login and see a list of your “DMs” and start a voice call as well. However this prototype needs to be rewritten, most parts at least if not everything.

prototype screenshot

License of the software

The Matrix app source code will be available to all for free on some site like Github under GPLv3. It will be available on BananaHackers store as well.

Stages

So I split the development in the following stages:

Stage 1 -once 200 dollars is reached, summer vacation has started and I have bought a Nokia 800 Tough- will be an app with the following features:

  • Login with username and password is possible only.
  • You could see a list of DMs
  • Invites are auto-accepted
  • Possible to start a voice call with someone or receive voice call as long as the app's running
  • The app will be extensible letting us add other Matrix features later easily

Stage 2 -once 300 dollars is reached and stage 1 has been done, includes push notification for incoming calls enabling users to receive calls even if the Matrix app is not running. This requires some server software as well as some modification in the KaiOS app. Also invites are no longer auto-accepted.

Finally Stage 3 adds video call functionality once 350 dollars is reached and also tries to improve the UI as well as performance.

Limitations

Due to memory hungry design of matrix-js-sdk, users might not be able to use the app with their account if they have joined large rooms.

Another thing is that I won't be sending the app to KaiStore due to their foolish rules for Voice over IP apps.

Tech stack

Tech stack for the KaiOS app, most likely, will be either ReactJS or InfernoJS as well as Adrian's KaiUI. And for the push notification server software it would be either Python or Rust which is preferred.

Entering the campaign

So if you want to enter the campaign, send cryptocurrency to one of the addresses below and remember even small amounts would help: EDIT: The campaign has ended, check the same blog for status of the project.

BTC: 1Lv3FRzqcG7GhLcAMrHDwXAN6dMbfLu78g
BCH: 1AGk1t3oCxeTqYX9kAUJHFULdP32D4d6d6

Check how much was raised here for Bitcoin and here for Bitcoin cash.

You don't have crypto? No problem you could buy some from places like Coinbase or Kraken or other exchanges.

According to commits on kaiscr NotABug Git repository about 9 months after I had done my first commit in the repository, I started another serious research for finding a better and high performance way for mirroring the screen of a KaiOS device. Current kailive.py script works by taking multiple screenshots from the screen using kaiscr.py and can go up to 10-12 FPS. However most of times, the FPS rate is under 10 and forces high resource usage to the device making many heavy apps slow if you want to run the app and mirror the screen at the same time.

My guess was that, the problem is with encoding raw image to PNG on the device for each frame because screenshot file type is PNG. According to what fabrice said in #b2g:mozilla.org(A Matrix room), it should also be able to give JPG instead of PNG but that doesn't solve the problem.

If I was correct, I should be able to achieve good performance by somehow taking raw screenshots. My early researches, shortly after I had done with kailive.py, was more focused on finding a low level way using Android libraries or even the Linux kernel to read raw pixels from GPU or some buffer somewhere. But I had no such a low level experience so I failed finding a way. I tried reading from /dev/graphics/fb0, the Framebuffer, using fbcat, too. But the result was a blank image(all black) with the resolution equal to screen's resolution. And I had other evidences leading me to conclude the fact that Framebuffer is not used for displaying stuff on these devices, like Framebuffer is slow and my bananaphone is capable of running high FPS games(even 30 is high for FPS as far as I know).

I had left this for a long time without any further research, till one week ago(or two). This time my approach was researching a higher level of the system, Gecko. Fortunately after “grepping” and tracing the source code of Gecko for 8110(available on Nokia website at the time of writing this post) I had found how does the System app take screenshots and output them as PNG.

It draws the whole window on a canvas, saves the content of that canvas to PNG and sends it through debugger-socket. Soon after that I found a way to export raw RGB data rather than PNG and sent it to a TCP server listening on my laptop. But did I succeed? The answer is No!

First I tried 30 fps but neither the device was fast nor I got the wanted FPS rate. The same goes for 20 fps... This means that my guess was simply wrong. The only thing which I have not tried is captureStream() but I have no hope it will work as expected.

I have written a short report with code snippet in the wiki which you can read here: https://wiki.bananahackers.net/en/development/screen-mirroring#taking-screenshots-without-encoding-and-decoding-to-or-from-png