Skip to main content

 

The wiki for MoeNavigatorEngine now contains documentation on how to contribute to the project. It also gives you a basic explaination of how the network stack works and how parsers work in MoeNavigatorEngine so that you don't have to look into the code anymore to figure that out:

https://codeberg.org/moenavigator/moenavigatorengine/wiki/index

If you have cloned the source code repository, you can download the whole content of the wiki by invoking "git pull" followed by "git submodule init" and "git submodule update". The wiki content is in the "doc" folder.


More documentation on the different parts of MoeNavigatorEngine will follow. Any specific component you are curious about?


# # # # # #

 

Comparison of the source code of four different XMPP clients


In a Matrix vs. XMPP discussion a few months ago, somebody mentioned that the XMPP clients look old-fashioned, not like a modern messenger. I was wondering if it might be the source code that prevents people from beautifying well-known desktop XMPP clients, if such efforts seem necessary. So I took a brief look at the source code of Gajim, Kopete, Pidgin and Psi to see if it is well-structured, documented (source code documentation) and if the source code is organised in a way that would make a redesign of the GUI or improvements in general easy. The results:


1. Gajim:
The GUI is defined in XML files (probably for GTK), so it should be it easy to redesign the GUI itself with a designer. Gajim uses Python which is fairly well-known and can be used for big projects like an XMPP client. The source code could need more structure to make it more clear which files belong to the GUI and which belong to other parts of the program. Lots of source files lack source code documentation.


2. Kopete:
The GUI definition is placed in XML files (for Qt), so redesigning the GUI is probably not an issue. The files in the "kopete" subdirectory lack source code documentation, some files in the "plugins" and "protocols" subfolder have some documentation. The files I viewed from the "libkopete" subfolder have lots of documentation: The classes and their methods are well documented in the header files and even in the source files you will find hints what a block of code is doing. The code in general is well enough structurised to quickly find out where to look for a specific source file. Using Qt-C++ as programming language surely is helpful with such a big project.


3. Pidgin:
If you know that Pidgin's core functionality comes from the libpurple library, then it is easy to guess where to look for internal stuff: the "libpurple" subdirectory. The GUI code is placed in the "pidgin" subdirectory and the definitons in a subdirectory called "resources" which I find unintuitive ("gui" would be better). Pidgin is written in C. The source code in both the "pidgin" and the "libpurple" directory is very unordered since most of the source files are directly placed below these folders. Only protocols, plugins and some other parts get their own subfolders. Header files often have function documentation blocks, but in the corresponding source files you have to search for comments that explain what the code is doing. I can imagine that writing such a big project entirely in C can make it complicated for people new to the project to write good code.


4. Psi:
Where is the gui code? Ah, it is in the "src" subdirectory, together with the rest of the source code. At least the GUI definitions are stored in XML files (for Qt). But there is only this flat directory with all source files. This can make it hard to figure out where to look for specific functionality. The source code is in Qt-C++, but lacks documentation, both in header an source files. The coding style differs and seems to be chaotic sometimes. In general, I consider that bad code quality.



Conclusion:
Pidgin and Psi have big source code quality problems that can make it hard for new people to improve things. Gajim and Kopete on the other hand seem to be good candidates for that. But both could require more source code documentation.


# # # # # # # # #

 

MoeNavigator update: The engine got a new network stack


In short: The network stack in MoeNavigatorEngine has been cleaned up and has become more flexible. The tools for the engine have been updated, MoeNavigator got a bit of maintenance.


The refactorisation of the network stack in MoeNavigatorEngine (MNE) has progressed far enough so that the new code has been merged back into the master branch.

The new stack relies on a network handler that interfaces with the network functionality of the operating system and request objects that pass themselves to the network handler when the request starts. The difference to the old stack is that there is no network protocol handler anymore that passes data. Instead, the network handler sends data to the request object's buffer which is then passed to the "other side" of the web browser engine that processes the data in one of its parsers. To avoid polling, mutexes are used as semaphores which get locked when data have been read and unlocked when data is available. The new network stack should also be able to handle asynchronous requests, but that isn't implemented yet.

The protocol support is the same as with the old network stack: HTTP only. But in the new stack, the Request class provides an abstraction layer where protocols are just specialisations of the base Request class. Adding support for other protocols (like gopher) should have become easier with the new network stack.

The HTTP support has improved in the new stack since the header fields are not limited to a certain set of fields. All header fields are treated equal in request and response headers as long as they are valid. The HTTPRequest class doesn't care what's inside the fields, its only repsonsability is to put together the request header, parse the response header and read the response data. I have verified that reading big binary files works using the mneget tool (from the moenavigatorengine-tools repository). To send data via HTTP POST or PUT, the HTTPRequest class needs some more code to send big chunks of request data.

To add TLS support, a layer between the network handler and the request is necessary. How it shall be named is not yet decided, but it will probably implement the NetworkHandler interface so that all methods for unencrypted network access can be available for encrypted network access as well. This would enable support for HTTPS and could add TLS support for all other protocols the engine supports so that theoretically there could be something like "gophers" (gopher secure), once gopher is implemented.

Besides the work on the engine, I updated the tools for it and got mneget, a wget-like program, into a working state. MoeNavigator itself got a little bit of maintenance and an updated about dialog. With the big progress in MoeNavigatorEngine this week, the days of WebKit as default engine in MoeNavigator are numbered.


About

MoeNavigator is a web browser written from scratch in C++. It uses its own engine called MoeNavigatorEngine (MNE). There is also a collection of tools for the engine. The source code is licensed under the terms of the GNU General Public License v3. The repositories are on @Codeberg.org:

- https://codeberg.org/moenavigator/moenavigator
- https://codeberg.org/moenavigator/moenavigatorengine
- https://codeberg.org/moenavigator/moenavigatorengine-tools


# # # # # # # #

 

MoeNavigator progress


After months, I have continued to work on the refactorisation of the network stack of MoeNavigatorEngine, the web browser engine of MoeNavigator.

In the new network stack, the Request class (and especially the derived HTTPRequest class) get a central role in handling request and response data. They interact directly with the network handler on the one side and the engine core on the other side. The architecture should be able to allow synchronous and asynchronous network requests in the future. At the moment, the code to read data from an URL in MoeNavigatorEngine reads response data in chunks using a synchronous approach.

During implementation of the new network stack, I thought about including libcurl, but it couldn't be integrated into the new network architecture, since it doesn't allow separating transfer protocols like HTTP from the underlying network functionality. For a web browser engine, I think it is necessary to have the full control over each bit of HTTP data that is sent or received. So the network stack of MoeNavigatorEngine continues to use POSIX sockets directly without a library in between and protocols are implemented in classes derived from the Request class.

There are still some tasks to be done before the network refactorisation is finished. The progress can be seen here: https://codeberg.org/moenavigator/moenavigatorengine/issues/9

The "network-refactorisation1" branch in the git repository contains the current state of the refactorised network code. It is still unstable, method signatures and interface definitions may still change.


Info
MoeNavigatorEngine is my attempt to write a web browser engine.

Project page at codeberg: https://codeberg.org/moenavigator/moenavigatorengine

# # # # #
 
Thinking about the new network architecture, it seems that middleware modules between a request and the network handler can easily be added, especially if the middleware implements the NetworkHandler interface. Support for TLS could be added by telling Request objects to interact with a TLS-NetworkHandler which in turn interacts with the "real" network handler. Of course, malicious plugins could hook themselves between the TLS-NetworkHandler and the Request object and read all data, but then again, you should know what you are doing when installing plugins.
 
Built-In TOR support would be possible by adding a middleware module between the TLS middleware and the "real" network handler.

 

Schlimm project update: releases planned


I have assigned tickets to milestones representing the next releases of Schlimm (simple configurable human login management module, my fork of SLiM).

There is still some cleanup and abstraction work to be done before the release of version 1.4.0:

https://codeberg.org/ncc1988/schlimm/milestone/210

Version 1.5.0 will introduce new features:

https://codeberg.org/ncc1988/schlimm/milestone/211


Any help is welcome. The code repository and the bug tracker are on @Codeberg.org :

https://codeberg.org/ncc1988/schlimm


# # # # #

 

libcuwte progress: it renders basic jinja2 templates again, but with designed code


The original libcuwte source code was able to render a basic jinja2 template with placeholders in it using proof of concept code. I now converted the code into proper code after I thought of a design for it.

Now the evaluation of if-statements in Jinja2 has to be done. When if-statements can be evaluated, other Jinja2 statements (for, foreach, swich, while, ...) can be added quickly since they can be converted to if-statements. The next template language to be implemented will probably be ERB.

libcuwte source code: https://codeberg.org/ncc1988/libcuwte

libcuwte wiki (currently only a feature matrix): https://codeberg.org/ncc1988/libcuwte/wiki

# # # # # #

 

Coding during corona - suggestions for projects?


I want to continue writing software in my spare time during the corona crisis. But which project is the most important or most valuable under the current circumstances so that I should focus on it?


There is a list of my active projects I made a few days ago:

https://alt-f4.der.moe/display/fa2bb979-165e-6e10-e25b-383076357160

There are some more projects I started:

https://codeberg.org/ncc1988/


Besides my personal projects, I'm open for suggestions regarding the support of other free software (FLOSS) projects by programming for them. Projects written in C++ or PHP, without the requirement of signing up on non-free platforms to be able to contribute, are preferred.

What do you suggest?


# # # # # # #

 

Software and hardware projects I'm currently working on


MoeNavigatorEngine


My web browser engine written from scratch since 2011. Currently I'm rewriting the network components for better HTTP request handling. The current state of the rewrite is in the "network-refactorisation1" branch of the source code.

Project page: https://codeberg.org/moenavigator/moenavigatorengine

libcuwte


Web template rendering in C++ using common template languages like Jinja2. The project just started but you can already replace variables in Jinja2 templates with it. Check out the test1 program in the source code repository for a demonstration.

Project page: https://codeberg.org/ncc1988/libcuwte

Schlimm


My fork of the SLiM desktop manager. Currently I'm refactorising the code, translating it to modern C++ and making it possible to use schlimm without any configuration file.

Project page: https://codeberg.org/ncc1988/schlimm

open-system-compact-cassette


This is a long-term hardware project where I'm trying to develop circuits and microcontroller code for compact cassette hardware to be able to record and playback digital music from a cassette using modern audio codecs like Speex and Opus. The current task is to find out how to connect GNURadio blocks correctly to input and output data at an acceptable rate using a sound card.

Project page: https://codeberg.org/ncc1988/open-system-compact-cassette

analogtape-utils


This is a collection of tools that help with converting media files for recording compact cassettes and VHS tapes using a computer. The encode_cassette.sh script for compact cassettes handles things like audio normalisation and pitch adjustment for tape recorders that don't record in standard speed. The encode_vhs script compiles all the source video files into one big video file using a Makefile for multitasking. The collection gets new scripts whenever I find a way to automate a part of the media conversion process.

Project page: https://codeberg.org/ncc1988/analogtape-utils

# # # # #

# # # # # #

 

Thoughts about: AUTHORS file and GDPR


I have created a fork of a GPL software project and would like to put all people who contributed source code into an AUTHORS file. I think that is unproblematic for those who left their name and mail addresses in the source code files since these personal data have already been published by them on a voluntary basis.

But I also would like to apply patches from Debian in my fork. Each patch contains the name and mail address of the author of the patch. According to the GPL, I must give credit to everyone who contributed source code but I'm unsure if that means I can just put someone into an AUTHORS file if they submitted a patch to "someone else", even if it happened in the public and the personal data were provided on a voluntary basis, too.

Are there examples on how other FLOSS software forks/folks deal with the GDPR in this matter?

# # # #

 

Friclicli update: moving to C++


I'm currently translating the source code of Friclicli (Friendica CLI client) from C to modern C++. Some code parts have already been rewritten and have been shortened by that due to the use of std::string instead of char arrays. The code translation process happens in the "move-to-cpp" branch to leave the master branch of the friclicli repository in a usable state.

At the moment there are linking errors because of undefined functions or functions whose definitions have been converted to class methods. It will take some time until the translation process has finished and I can continue to add new functionality to friclicli.

The source code repository is at codeberg: https://codeberg.org/ncc1988/friclicli


# # # # # # #
 
I'm creating tasks on Codeberg for the C++ translation. The tasks are all placed under the following milestone so that you can view the progress: https://codeberg.org/ncc1988/friclicli/milestone/142

 

Now that I have a Friendica account again, I also have a motivation to continue work on friclicli (Friendica CLI client): https://codeberg.org/ncc1988/friclicli

Before I continue to implement features, the source code should be converted from C to modern C++ to make it more maintainable.

# # # #