BananaHackers blog

Reader

Read the latest posts from BananaHackers blog.

from farooqkz

Matrix logo

#matrix #voip #voicecall

The story

It had been a very long time which I wanted to start developing a Matrix client with voice call support for KaiOS. Before that I had tried developing a Jitsi Meet app for KaiOS letting KaiOS users have group voice chat with their feature smartphones, which failed due to their ToS.

What is Matrix?

Matrix is an open and federated protocol for chatting which supports End to End Encryption, 1 to 1 voice and video calls(support for group calls is experimental). Is it also possible to integrate other chat applications like XMPP, Telegram, Discord, IRC, etc into Matrix using bridges.

What am I going to do?

I want to create a Matrix call only app for KaiOS letting users of this platform to use 1 to 1 calls, especially voice call. I need at least 350 US dollars to work on it as a job about 15 hours a week and to get a Nokia 800 Tough which costs me about 120 dollars in my country.

Note that I've got a 8110(aka the bananaphone) but its keypad is almost dead.

Donators' name as well as the amount they've donated shall be written in Credits/About tab of the app visible to everyone who uses the app. If you want your name to be written you MUST email me your crypto public key before making the transaction

hapu [at] riseup [dot] net

I already have written a prototype of a Matrix app which let's you login and see a list of your “DMs” and start a voice call as well. However this prototype needs to be rewritten, most parts at least if not everything.

prototype screenshot

License of the software

The Matrix app source code will be available to all for free on some site like Github under GPLv3. It will be available on BananaHackers store as well.

Stages

So I split the development in the following stages:

Stage 1 -once 200 dollars is reached, summer vacation has started and I have bought a Nokia 800 Tough- will be an app with the following features:

  • Login with username and password is possible only.
  • You could see a list of DMs
  • Invites are auto-accepted
  • Possible to start a voice call with someone or receive voice call as long as the app's running
  • The app will be extensible letting us add other Matrix features later easily

Stage 2 -once 300 dollars is reached and stage 1 has been done, includes push notification for incoming calls enabling users to receive calls even if the Matrix app is not running. This requires some server software as well as some modification in the KaiOS app. Also invites are no longer auto-accepted.

Finally Stage 3 adds video call functionality once 350 dollars is reached and also tries to improve the UI as well as performance.

Limitations

Due to memory hungry design of matrix-js-sdk, users might not be able to use the app with their account if they have joined large rooms.

Another thing is that I won't be sending the app to KaiStore due to their foolish rules for Voice over IP apps.

Tech stack

Tech stack for the KaiOS app, most likely, will be either ReactJS or InfernoJS as well as Adrian's KaiUI. And for the push notification server software it would be either Python or Rust which is preferred.

Entering the campaign

So if you want to enter the campaign, send cryptocurrency to one of the addresses below and remember even small amounts would help:

BTC: 1Lv3FRzqcG7GhLcAMrHDwXAN6dMbfLu78g
BCH: 1AGk1t3oCxeTqYX9kAUJHFULdP32D4d6d6

Check how much was raised here for Bitcoin and here for Bitcoin cash.

You don't have crypto? No problem you could buy some from places like Coinbase or Kraken or other exchanges.

 
Read more...

from affe-null

In the last post I wrote about the #ApiDaemon. In this post, I will show that it is possible to add “add-on” services to the daemon, and also explain what else the daemon does.

Remote services

Remote services are like plugins for the API daemon. Unlike normal services, they don't have rust libraries that are compiled into the api-daemon binary, but instead run as a separate “child” daemon. When needed, the API daemon starts the child daemon with the environment variables LD_LIBRARY_PATH set to the remote service's path and IPC_FD set to a file descriptor number that is used for IPC between the API daemon and the child daemon.

General structure

Remote service daemons are located in subdirectories of /data/local/service/api-daemon. The subdirectories are named after the service. Each subdirectory should contain at least a daemon binary but can also have a valid_build_props.txt file. Remote services also need a JS library in the http_root. For example, the wavoip2 service contains the following files:

/data/local/service/api-daemon/remote/wavoip2/daemon
/data/local/service/api-daemon/remote/wavoip2/libc++_shared.so
/data/local/service/api-daemon/remote/wavoip2/libwa_voip.so
/data/local/service/api-daemon/remote/wavoip2/valid_build_props.txt
/data/local/service/api-daemon/http_root/api/v1/wavoip2/service.js.gz

The valid_build_props.txt file contains a whitelist of values of ro.build.cver. If the file exists, the service is only considered valid if ro.build.cver matches one of the lines and SELinux is in enforcing mode. wavoip2 is even more strict: Even if the daemon does start, it still checks for SELinux enforcing and ro.build.cver values. Because of this, WhatsApp calls don't work on rooted or soft-rootable devices. However, you can spoof SELinux enforcing with the following commands:

echo -n 1 > /data/local/tmp/enforce
mount -o bind /data/local/tmp/enforce /sys/fs/selinux/enforce

And if you then run setprop ro.build.cver QC_8905_3_10_49_SEC_1, wavoip will start successfully.

I haven't figured out the IPC protocol yet, but looking at the API daemon's source code will probably help: https://github.com/kaiostech/api-daemon

KaiAds

The API daemon also serves the #KaiAds SDK for privileged apps. This is probably used because of CSP restrictions that apply only to privileged and certified apps. Newer versions of the SDK provide a “SDK loader” that decides whether to load the real SDK from kaiads.com or http://127.0.0.1:8081/sdk/ads.

Telemetry

Another thing which the daemon does is telemetry. I first noticed this when I tried enabling SELinux enforcing to get wavoip working. After running setenforce 1, my phone suddenly became very slow, and when I looked into logcat, there were a bunch of lines saying that the telemetry process couldn't read from /proc/kmsg!!! Later I found out that this process belongs to the API daemon. This means that the API daemon reads the kernel log messages for telemetry purposes.

Also, I found the following lines in the daemon's configuration file:

[telemetry]
enabled = true
send_app_opened_events = true
token_uri = "https://api.kaiostech.com/v3.0/applications"
metrics_uri = "https://api.kaiostech.com/v3.0/apps/metrics"

so the daemon also sends “app opened” events to KaiOS.

All of this can be disabled by setting enabled to false. But KaiOS also provides development versions of the API daemon that have enabled set to false by default: https://packages.stage.kaiostech.com/base_23/api-daemon-1.4.26.zip. I will write about the packages.*.kaiostech.com domains in the next post.

 
Read more...

from affe-null

You might have already heard that #WhatsApp uses some undocumented APIs that are built into the #KaiOS system. These APIs are powered by the api-daemon, a daemon which lies at a lower level than Gecko. In this post, I will explain what I have found out about this daemon.

The api-daemon is written in #Rust, and provides several “services”. Each service has two parts, the JS client and the Rust library. The Rust library is linked into the api-daemon executable. The JS clients are accessible via HTTP at http://127.0.0.1:8081/api/v1/<service>/<script>.js and expose APIs in a lib_<service> variable. A pre-installed version of the daemon is available in /system/kaios, but it is copied to /data/local/service to allow updates without modifying the system partition.

Using the APIs

To demonstrate the usage, let's create a simple app called “Daemon Test”. We will use the telephony service. Although it is meant to be used to get and set the call state, it seems that real calls have no effect on the reported state. However, the state might reflect ongoing WhatsApp and other VoIP calls.

To use the APIs on KaiOS 2.5, we need the external-api permission:

manifest.webapp

{
  "name": "Daemon Test",
  "launch_path": "/index.html",
  "description": "An app to demonstrate api-daemon functionality",
  "type": "privileged",
  "permissions": {
    "external-api": {}
  }
}

index.html

<!DOCTYPE html>
<html>
  <body>
    <div id="callstate">Loading...</div>
    <script src="http://127.0.0.1:8081/api/v1/shared/core.js"></script>
    <script src="http://127.0.0.1:8081/api/v1/shared/session.js"></script>
    <script src="http://127.0.0.1:8081/api/v1/telephony/service.js"></script>
    <script src="daemon-test.js"></script>
  </body>
</html>

We can now use these APIs as described in https://developer.stage.kaiostech.com/docs/sfp-3.0/09.migration-from-2.5/next-new-apis/daemon-api/telephony/daemon-api-telephony/, except that we will need to replace localhost with localhost:8081. Note that the documentation is for KaiOS 3.0, but on 2.5 only the telephony, libsignal (for WhatsApp) and tcpsocket services are available. You can also look at the api-daemon source code, which is also only for KaiOS 3.0.

First, we need to set up a sess variable that stores the api-daemon session:

var sess = new lib_session.Session();

Then, we write some handlers that get called when the session connects/disconnects:

function onsessionconnected(){
  lib_telephony.TelephonyService.get(sess).then(function(service){
    /* handle session connect */
  });
}
function onsessiondisconnected(){
  /* handle session disconnect */
}

And after that we try connecting.

sess.open('websocket', 'localhost:8081', 'secrettoken',
  {onsessionconnected, onsessiondisconnected}, true);

Here is the final code:

daemon-test.js

window.onerror = function(e){
  // Error handling
  alert(e);
}

var sess = new lib_session.Session();
var el_state = document.getElementById('callstate');
var toggleCallState = null;

function callStateChangeCallback(state){
  el_state.textContent = state;
}

// Set up session callbacks
var onsessionconnected = function(){
  alert('Session connected');
  // Get an instance of the telephony manager
  lib_telephony.TelephonyService.get(sess).then(function(manager){
    alert('Got Telephony manager');
    try {
      // Show the call state
      manager.callState.then(function(state){
        el_state.textContent = 'Call state: ' + state;
      });

      // Show it again if it changes
      manager.addEventListener(manager.CALLSTATE_CHANGE_EVENT, function(state){
          el_state.textContent = 'Call state: ' + state;
      });

      toggleCallState = function(){
        // Get the call state
        manager.callState.then(function(state){
          // Set the call state
          manager.callState = state ? 0 : 1;
          alert(state ? 'Idle' : 'Calling');
        });
      }
    } catch(e) {
      alert(e);
    }
  }).catch(function(e){
    alert('Failed to get Telephony manager: ' + JSON.stringify(e));
    toggleCallState = null;
  });
}
var onsessiondisconnected = function(){
  alert('Disconnected');
  toggleCallState = null;
}

// Initialize the api-daemon session
sess.open('websocket', 'localhost:8081', 'secrettoken',
  {onsessionconnected, onsessiondisconnected}, true);

window.addEventListener('keydown', function(e){
  if(e.key == 'Call'){
    // Toggle the state if 'Call' is pressed and the session is ready
    if(typeof toggleCallState === 'function') toggleCallState();
  }
});

There is a lot more to this. I will also make a wiki page, and some more posts about things such as the updater-daemon and remote services.

More tags: #ApiDaemon

 
Read more...

from farooqkz

According to commits on kaiscr NotABug Git repository about 9 months after I had done my first commit in the repository, I started another serious research for finding a better and high performance way for mirroring the screen of a KaiOS device. Current kailive.py script works by taking multiple screenshots from the screen using kaiscr.py and can go up to 10-12 FPS. However most of times, the FPS rate is under 10 and forces high resource usage to the device making many heavy apps slow if you want to run the app and mirror the screen at the same time.

My guess was that, the problem is with encoding raw image to PNG on the device for each frame because screenshot file type is PNG. According to what fabrice said in #b2g:mozilla.org(A Matrix room), it should also be able to give JPG instead of PNG but that doesn't solve the problem.

If I was correct, I should be able to achieve good performance by somehow taking raw screenshots. My early researches, shortly after I had done with kailive.py, was more focused on finding a low level way using Android libraries or even the Linux kernel to read raw pixels from GPU or some buffer somewhere. But I had no such a low level experience so I failed finding a way. I tried reading from /dev/graphics/fb0, the Framebuffer, using fbcat, too. But the result was a blank image(all black) with the resolution equal to screen's resolution. And I had other evidences leading me to conclude the fact that Framebuffer is not used for displaying stuff on these devices, like Framebuffer is slow and my bananaphone is capable of running high FPS games(even 30 is high for FPS as far as I know).

I had left this for a long time without any further research, till one week ago(or two). This time my approach was researching a higher level of the system, Gecko. Fortunately after “grepping” and tracing the source code of Gecko for 8110(available on Nokia website at the time of writing this post) I had found how does the System app take screenshots and output them as PNG.

It draws the whole window on a canvas, saves the content of that canvas to PNG and sends it through debugger-socket. Soon after that I found a way to export raw RGB data rather than PNG and sent it to a TCP server listening on my laptop. But did I succeed? The answer is No!

First I tried 30 fps but neither the device was fast nor I got the wanted FPS rate. The same goes for 20 fps... This means that my guess was simply wrong. The only thing which I have not tried is captureStream() but I have no hope it will work as expected.

I have written a short report with code snippet in the wiki which you can read here: https://wiki.bananahackers.net/en/development/screen-mirroring#taking-screenshots-without-encoding-and-decoding-to-or-from-png

 
Read more...