Project

General

Profile

News

vanilla: LLVM 9.0.0 toolchain available

Added by David Demelier about 1 month ago

The toolchain LLVM 9.0.0 has been successfully upgraded for Vanilla Linux. This makes Vanilla Linux one of the first distributions to provide it.

What's most exciting feature is the ability to build the Linux kernel with clang.

The current repository will be updated once the final passes of LLVM 9.0.0 have been fully rebuilt along with musl and busybox. Stay tuned.

vanilla: 0.2.0

Added by David Demelier 2 months ago

Vanilla Linux 0.2.0 is now available for download.

About this release

This version adds many new packages with now a total of ~830 packages.

Notable additions:

  • A large amount of KDE frameworks were incorporated.
  • Most of the programs that are PAM compatible have appropriate working configuration files.
  • The complete aspell package list has been added.
  • A large amount of libretro and RetroArch packages has been added.
  • More audio and video codecs.
  • A web browser is now available based on webkit2gtk: midori.
  • Almost all Qt 5 modules are available.

Notable changes:

  • The new FHS, Vanilla Linux no longer follow the LSB recommandation and has a simpler hierarchy. No /usr/bin, /usr/sbin or /sbin, everything is in /bin.
  • LibreSSL replaced OpenSSL.

Improvements to vpk

Here is a list of changes to the Vanilla Linux vpk package manager.

  • Support of dependencies. This is still considered experimental and some packages may have forgotten dependencies yet. This will be fixed properly in 0.3.0. For now, please verify if there are missing dependencies after installation.
  • A new vpk add command that install local packages without dependencies, mostly used for scripting.
  • After installation of packages, vpk will print:
    • package message if specified (install bluez for example).
    • protected files that need update (install etc package, edit one the files it provides, then reinstall etc for an example).
  • The vpk build now build either only the binary or only the source package if one of -s or -b was specified.
  • An experimental command vpk bulk that download the source package and rebuild their dependencies and the package itself in a clean chroot.
  • Several predefined hooks are installed and ran after installation of packages (fontconfig, gtk-update-icon-cache and update-mime-info).
  • Minor performance improvements
  • The option -r was removed from every command. In contrast to the 0.1.0 release, the recommended way to download and use Vanilla Linux is to extract the minirootfs file instead. See below.

Testing

Download the minimal image and extract it under a subdirectory.

Example

$ mkdir vanilla-0.2.1
$ cd vanilla-0.2.1
$ wget http://releases.malikania.fr/vanilla/0.2/images/Vanilla-Linux-0.2.1-amd64-minirootfs.tar.xz
# tar xvJpf Vanilla-Linux-0.2.1-amd64-minirootfs.tar.xz
# cp /etc/resolv.conf etc/
# mount --rbind /dev ./dev
# mount --make-rslave ./dev
# mount --rbind /sys ./sys
# mount --make-rslave ./sys
# mount --rbind /proc ./proc
# mount --make-rslave ./proc
# chroot . env -i sh -l
# export TERM=xterm-256color (recommended)
# export PATH=/bin:/local/bin
# vpk search supertux

Upcoming for 0.2.0

A XFCE desktop bootable QEMU image with will be created and uploaded once ready in the http://releases.malikania.fr/vanilla/0.2/images, stay tuned.

What's next for 0.3.0

The main priority for 0.3.0 is to consolidate existing packages and make sure they are the cleanest and well configured. The key idea is to finalize the vpk bulk command and start using it for every new package. Also, this command will verify the built package to make sure no dependencies are missing after installing it.

The most important feature planned is the vpk upgrade command. For now, there is no way to upgrade packages because I spend much more time adding new packages than updating existing. But it's time to incorporate this major command since its one of the last important bits to add to it.

irccd: Removing BitBucket official mirror

Added by David Demelier 3 months ago

After the sad decision that Atlassian did about removing Mercurial support. I announce that the official irccd mirror will be removed today along with my account on it.

Also, I recommend people to never trust companies and hosting services. I'll no longer create official mirrors on proprietary hosting services starting from now.

Don't hesitate to advice opensource hosting services that have Mercurial support if you are aware of some.

vanilla: 0.1.0

Added by David Demelier 8 months ago

Vanilla Linux 0.1.0 is now available for download.

About this release

This version includes vpk 0.1.0 which supports:

  • Building packages vpk build,
  • Downloading packages, source packages and distfiles vpk download,
  • Installing packages (no dependencies yet) vpk install,
  • Removing packages vpk remove,
  • Querying package list vpk list,
  • Querying package information vpk info,
  • Updating manifest file vpk update.

The vanilla sources contain around 300 packages with several options.

As bootloaders, grub and efibootmgr are provided, lilo is available too but fails to run on my system.

No Linux kernel is provided yet so you will need to build it yourself for the moment.

Even though planned for 0.2.0, most of the X.Org distribution and wayland packages are already available for download!

Testing

To install in a chroot or a directory (/tmp/vanilla in this example):

Warning: make sure you don't forget -r option otherwise you'll override your running system!

First, install vpk on the host. Use 0.1.0 version because the package format has changed.

# wget http://releases.malikania.fr/vpk/0.1.0/vpk-0.1.0.tar.xz
# tar xJf vpk-0.1.0.tar.xz
# make -C vpk-0.1.0 install

Install filesystem, etc in this explicit order.

# vpk update -r /tmp/vanilla
# vpk install -r /tmp/vanilla filesystem
# vpk install -r /tmp/vanilla etc

Then, install some core packages.

# vpk install -r /tmp/vanilla bash ncurses readline musl gettext zlib coreutils libcap acl attr
# vpk install -r /tmp/vanilla llvm clang lld libc++ libc++abi libunwind compiler-rt

Then you can chroot, first make sure /dev, /sys and /proc are mounted.

# mkdir -p /tmp/vanilla/{dev,sys,proc}
# mount --rbind /dev /tmp/vanilla/dev
# mount --make-rslave /tmp/vanilla/dev
# mount --rbind /sys /tmp/vanilla/sys
# mount --make-rslave /tmp/vanilla/sys
# mount --rbind /proc /tmp/vanilla/proc
# mount --make-rslave /tmp/vanilla/proc
# chroot /tmp/vanilla env -i bash --login
# export PATH=/usr/bin:/usr/sbin

Now, you're also be able to compile and build a program using clang and clang++. In the latter case, it will link against libc++, libc++abi and libunwind.

Notable packages:

  • vim 8.1
  • gcc 8.3.0
  • llvm 8.0.0 (with lld, clang, compiler-rt, libc++, libc++abi, libunwind)
  • bash 5.0
  • coreutils 8.30
  • cmake 3.13
  • mercurial 4.9
  • wayland 1.16
  • weston 5.0.0
  • mesa 18.3.5
  • gtk 3.24
  • musl 1.1.21
  • ncurses 6.1
  • perl 5.28
  • python 3.7 and 2.7
  • xorg-server 1.20

irccd: 3.0.0 date

Added by David Demelier 10 months ago

I've posted a news 8 months ago saying that 3.0.0 will come soon.

I apologize this didn't happen. It will, I take more time to improve any last bit to make this release as clean and perfect as possible. The hardest part is the C++ API, since 3.0.0 will include an official C++ API for developing plugins, I will need to take care of this compatibility during the 3.y.z lifetime.

And since I hate to keep bad details, I'm still reading and checking every single function that I could write in a more cleaner way to avoid cluttering the API with deprecated functions/types after 3.0.0 will be released. I also take the consistency very seriously. For example the term "host" and "hostname" were mixed in a very large amount of code. Some options were named "host" and some "hostname". I've fixed all that in 3.0.0 but I want to be sure all other things in C++ API, Javascript API, options, plugins, network API, irccdctl output will be absolutely perfect once 3.0.0 is released.

Unfortunately as the only one contributor and not having many feedbacks, I don't always see all these small typos or errors and need more time to improve irccd.

Anyway, irccd 3.0.0 will happen this year and I really want it to be a long term release with the best user experience.

irccd: Mercurial repository edition

Added by David Demelier about 1 year ago

The Mercurial repository will be publicly edited on Wednesday 07 November evening.

I've discovered that the tag 2.2.0 and its signature commit were created on the wrong branch and this requires manual history edition. This means that you should either download a fresh copy of the repository or to strip and update.

Warning: if you have local changesets, those will be removed.

# you need strip extension
hg strip -r 0
hg pull
hg up @

irccd: System specific features

Added by David Demelier over 1 year ago

While I have added those options in 2.0.0, I felt that it's not the purpose of a program to perform these operations. Even though there are common unix features, they also break unix philosophy of doing only one thing right.

UID and GID

Handling UID/GID switching from code requires number of checks. We need to check if setgid, setuid are available and with appropriate headers.

Some systems already have a daemon utility to run a program in background specifying a user to run as.

Exemple on FreeBSD:

daemon -u nobody irccd -v

Daemon

The daemonize process was very early in the previous code. And when debugging I was always specifying the -f flag to be sure irccd does not run in background. Unfortunately I forgot many times and I got multiple daemon trying to connect in many servers. This was quite frustrating even though I understand it's my own issue.

But as explained above, there is already a daemon tool to actually spawn a process into a daemon, there is also nohup or simple tmux for some people.

Daemon has also been considered deprecated by macOS and therefore a warning was issued at compile time that requires a special compiler flag to disable a warning and I hate to disable warnings.

In my opinion, it's the business of a service manager to spawn a process into the background and taking care of it. Systemd by default assume programs do not fork by themselves unless you specify Type=forking in the unit file.

PID

Irccd no longers write it's own process id into a file. For the same reasons above, I think it's the responsability of the service manager to track its process and be sure it's still running.

What does that change for me?

Not much.

Irccd 3.0.0 will come with a service file for FreeBSD and to configure the user/group of the service FreeBSD already used the convention program_user and program_group in the /etc/rc.conf. That means you will able to change its UID/GID like this:

irccd_enable="YES" 
irccd_user="nobody" 
irccd_group="nobody" 

for systemd, the unit file won't set a user/group by default for now because it's very hard to determine portable user and group across the distributions. But with systemd override support, you will be able to set User and Group properties.

KISS

Remember, the unix philosophy prefers to delegate as much as possible functions that could be handled by external tools. This changes has removed a high number of #ifdef and CMake checks, keeping the code cleaner.

The whole change has removed around 343 lines of code.

irccd: 3.0.0

Added by David Demelier over 1 year ago

Irccd 3.0.0 is coming soon!

The next major irccd version will be released in the next few months. While it's a major version step ahead 2.0.0, it has less more changes than the step between 1.y and 2.y. There are still exciting features that I'll show you now.

Native plugins

Native plugins are finally here!

You can now write a plugin in C++ in case of need. Even though I really encourage to write a Javascript plugin, one may need to use C++ for various reasons (e.g. networking).

A plugin needs to derive from plugin class and export a unique symbol name, that's it.

Example of very basic plugin:

#include <iostream>

#include <irccd/daemon/plugin.hpp>

class my_plugin : public plugin {
public:
    my_plugin()
        : plugin("translate-me", "")
    {
    }

    void on_load(irccd&) override
    {
        std::cout << "native plugin loaded!" << std::endl;
    }
};

// Note, one may need BOOST_SYMBOL_EXPORT on Windows though.
extern "C" my_plugin irccd_plugin_translate_me;

// And define it.
my_plugin irccd_plugin_translate_me;

Note: the symbol name is tied to the file name by default. For example, if you want to load a native plugin with id “translate-me”, a file translate-me.so|dylib|dll will be searched and the symbol irccd_plugin_translate_me must be exported (hyphens are translated to underscores).

Warning: remember that irccd has currently no safe way to protect itself for broken plugins, if your plugin crashes, irccd will too.

The exported IrccdDefinePlugin will help developers to build and install plugins along with their documentation easily.

Live rule editing

The irccdctl tool gained a set of rule editing functions. The following commands have been added:

  • irccdctl rule-add, add a new rule into irccd,
  • irccdctl rule-info, get information about a rule,
  • irccdctl rule-list, list all current rules,
  • irccdctl rule-edit, update an existing rule,
  • irccdctl rule-move, swap rule positions.

Example, listing rules

Listing rules with rule-list:

$ irccdctl rule-list 
rule:        0
servers:     
channels:    
plugins:     hangman 
events:      
action:      drop

rule:        1
servers:     localhost 
channels:    #games 
plugins:     hangman 
events:      
action:      accept

Editing an existing rule

With the same set as before, edit one and see:

$ ./bin/irccdctl -c irccdctl.conf rule-edit --add-channel "#staff" 1
$ ./bin/irccdctl -c irccdctl.conf rule-info 1
rule:        1
servers:     localhost 
channels:    #games #staff 
plugins:     hangman 
events:      
action:      accept

Tests, tests and tests

This version adds a very large number of unit tests. I wanted to make this version the most well tested as possible.

I have added unit tests for irccdctl commands, now I will guarantee that irccdctl output stays compatible between minor versions. This also ensure compatibility in irccdctl options and arguments. Error strings are tested as well to be sure that scripts can rely on irccdctl in a safe and compatible way. The network protocol API errors are tested as well, unit tests have been added to ensure compatibility if commands are not completely valid.

Tests your plugins

If you have used irccd in the 1.y era, you may remember the irccd-test command, it was removed in 2.0.0 for some reasons but is now available again. In bonus, it contains a basic completion support for event names (requires libedit library).

$ ./bin/irccd-test ../plugins/ask/ask.js
> onCommand localhost jean #staff Will I be rich?
localhost: connect
localhost: message #staff jean, Yes!!
> onCommand localhost jean #staff Are you sure?
localhost: message #staff jean, Definitely not!
> 

In this example, if you start typing onCom and complete with tab key, you will be completed through onCommand. For the moment, there is no completion on servers names but I'll try to add this later.

Note: as you can see, there is an onConnect event before the response, this happens because each time you invoke an event with a different server, a fake one is created if it does not exist.

Internal code change

Most of the development have been done to switch to Boost. I'm sorry if you feel disappointed of having boost dependency in irccd, but I was not able to provide perfect system compatibility by writing my own code in many places.

On the other hand, irccd does not use boost for everything, the following modules are used:

  • Asio,
  • Filesystem,
  • Process (unit tests only),
  • Timer (Javascript API only),
  • Format.

New tictactoe plugin

A brand new tictactoe plugin has been added. It allows two players playing tictactoe game on IRC. The plugin takes care of checking if users are in the channel and stay the lifetime of the game.

< markand> !tictactoe linkdd 
< irccd>   a b c
< irccd> 1 . . .
< irccd> 2 . . .
< irccd> 3 . . .
< irccd> markand, it's your turn
< markand> a 1
< irccd>   a b c
< irccd> 1 x . .
< irccd> 2 . . .
< irccd> 3 . . .
< irccd> linkdd, it's your turn
< linkdd> a 3
< irccd>   a b c
< irccd> 1 x . .
< irccd> 2 . . .
< irccd> 3 o . .
< irccd> markand, it's your turn

New joke plugin

A very simple joke plugin has been added.

The purpose of this plugin is to randomly choose a joke in a set of user defined jokes. It shuffles the list of jokes and pick one of the current list, therefore, you will not see the same joke until the full pile has been shown.

< markand> !joke
< irccd> keep calm and drop database
< markand> !joke
< irccd> francis: make me a sandwish
< irccd> lisa: do it yourself
< irccd> francis: sudo make me a sandwich
< irccd> lisa: ok

And since the plugin uses the templating system in irccd, it is possible to specify a set of jokes for a different server/channel.

Please note that there is no fallback in that method, if you start using templates in the configuration, you will need to symlink/create jokes for every channels you want to use.

Decomissionning libircclient

Unfortunately, the major part of irccd has left. I have decided to remove libircclient from our code and use a house code itself. Since IRC protocol is very easy, it was pretty quick.

The biggest problem of libircclient is the dependency on the `select(2)` system call. This was pretty hard to mix with the irccd main loop which uses Boost.Asio context object. Instead of creating a thread and complex code for retrieving data, I've created a simple class that does most of the IRC protocol.

It is not that big though:

$ cloc irccd/daemon/irc.*
       2 text files.
       2 unique files.                              
       0 files ignored.

github.com/AlDanial/cloc v 1.72  T=0.01 s (320.0 files/s, 137922.1 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C/C++ Header                     1             60            240            287
C++                              1             58             21            196
-------------------------------------------------------------------------------
SUM:                             2            118            261            483
-------------------------------------------------------------------------------

New path system

One of the biggest issue I had in the last two versions was the management of directories. Irccd had used a dynamic directory lookup when loading plugins, this confuse users and is hard to understand. For example, when the plugin ask was loaded, irccd searched different configuration and data directories and if it does not find one, it uses the system wide. You may think it's not a problem, but it is! What if a plugin needs to write some data? This is the reason you see in old documentation that you were needed to create a directory before using a plugin.

In irccd 3.0.0, there is no more “dynamic directories”. Instead, if irccd is started without any directories set, it uses the default system ones, period. There is no more an autodetection of user, system directories. Thus, if you want to use different directories, you may set the [paths] section.

There are two sections, the global [paths] and per-plugin [paths.name]. The first one is the defaults for everything and the second can be added to adjust settings for a specific plugin.

Example:

[paths]
cache = "/var/cache/irccd" 
config = "/etc/irccd/" 
data = "/var/irccd" 

If the plugin ask is started with this configuration, it will have the following directories set:

  • cache: /var/cache/irccd/plugin/ask
  • config: /etc/irccd/plugin/ask
  • data: /var/irccd/plugin/ask

And then, if you want, you can override any directory without the need to set all.

[paths]
cache = "/var/cache/irccd" 
config = "/etc/irccd" 
data = "/var/irccd" 

[paths.ask]
data = "/etc/irccd/jokes" 

Then, ask will have the following paths instead:

  • cache: /var/cache/irccd/plugin/ask (default from `[paths]`),
  • config: /etc/irccd/plugin/ask (default from `[paths]`),
  • data: /etc/irccd/jokes (overriden from `[paths.ask]`).

Miscellaneous

Another minor changes:

  • If irccd is built from a Mercurial repository, the irccd --version command shows the last revision,
  • If no identity is set into a [server] section, it tries to uses the system username by default,
  • The systemd service file is now first class citizen.
(1-10/35)

Also available in: Atom