Project devlog

Sep 30

Choosing the stack and setting up the frame

I wanted a self-hosted music bot that does not depend on anyone else's service. I chose Node.js and discord.js for the bot, Lavalink v4 as a separate audio server (the bot gives the orders, Lavalink decodes and pushes the audio into the voice channel), and Docker Compose to package both.

The first question that stopped me was what to paste into DISCORD_TOKEN. I thought it was the Public Key. Rereading the Developer Portal made it clear: the Token is the bot's password (never to be put on GitHub), while the Application ID is the public identifier used to invite the bot and register commands. So the .env file has been in .gitignore since the very first commit.

One small detail that cost time: slash commands registered "globally" show up very slowly, while registering per server with GUILD_ID makes them appear instantly. While testing, I used the second approach.

Sep 30

YouTube would not play

Playing a YouTube link gave This video requires login, then a signature function decoding error.

I tried, in turn:

• updating Lavalink's YouTube plugin to 1.18.2;
• OAuth login with a secondary account to get a refresh token;
• enabling extra fallback clients;
• turning off the VPN and running from my home IP.

The result was hit and miss. YouTube actively blocks this kind of access, and under its terms of service it is not something a product should be built on anyway.

Sep 30

Local music and the “Unknown title” problem

My music files have no tags, so the /local autocomplete was full of "Unknown title", which made picking a track very hard.

I wrote a library scanner: it reads tags with music-metadata; if they are missing, it guesses the artist and title from the file name (patterns like 01-track-name, Artist - Title) and from the folder; search ignores Vietnamese diacritics; and the cache is keyed on modified date and file size so later scans only read new files.

That led to /nhaajt: shuffle-play the whole folder, reshuffle when the round ends, and never stop until you press stop. I use a weighted shuffle (tracks with a 👍 come up sooner) and spread tracks by the same artist far apart, because a purely random shuffle often puts two songs by the same singer back to back.

Oct 1

From working to usable

The bot could play, but it was not something other people could use yet. My own checklist: a DJ role and default volume per server, skip votes when the room is busy, a fair queue (one turn each), /bump, 24/7 mode, a "Now playing" card built with Components V2, a music quiz, stats and Wrapped, and lyrics that follow the singing.

Where I got stuck:

• Restarting the bot lost the queue. I save a snapshot of the queue when a new track starts and when the bot shuts down. The bug I hit: restoring a position beyond the track length made Lavalink reject it, so I had to clamp the position to the allowed range.
• Health checks. I write the "still alive" signal to RAM (/dev/shm) instead of disk, so that later, on an SD card, it does not cause wear.
• The Wrapped card. I draw an SVG and convert it to PNG with @resvg/resvg-js; the container has no fonts, so I bundle open-source fonts with it.

From here on every feature ships with tests (node:test). By this point there were 131 tests.

Oct 1

Data from other people is never harmless

Track titles come from file tags, and tags can be set by anyone. A title like [click here](http://…) or <@id> would render as a fake link or ping someone in the bot's message.

While testing I found that discord.js's escapeMarkdown does not escape the characters [ ] ( ) < >. I wrote safeText and cleanMeta (stripping control characters, text-direction override characters and markdown-style links) and added a test for each case. Lesson: every string that comes in from outside (file tags, usernames, suggestion text) has to be sanitized before it is displayed.

Oct 1

An idea I decided not to build

I once came up with letting members submit YouTube links, with the bot using an mp3 converter site to download them into the music library, so the library would grow through its users. It sounded great.

But the more I thought about it, the more I saw three problems: copying copyrighted music and playing it to a whole server is infringement; my repo and website are public, so they would be very easy to take down; and technically it is unreliable too (those sites have no API, and YouTube blocks them just like the error on Sep 30).

The alternative I chose:

• /contribute: users submit their own files, and the bot owner approves each file before it enters the library. Off by default; only accepts Discord CDN URLs; limits on size and duration; duplicates blocked by hash; stored file names are chosen by the bot, never the user's (to avoid ../-style paths).
• /request: only records a track name or link in a wishlist, and never opens or downloads the link. When the track shows up in the library, the bot messages whoever asked.

Oct 1

The website, Vercel and the first 404

I built an introduction website in an engineering-drawing style (cyanotype blue paper), pushed the repo to GitHub (public) and deployed it on Vercel.

The page returned 404: NOT_FOUND. The cause: Vercel builds from the repo root, while the site lives in web/. I fixed it with a vercel.json at the root that sets the output directory to web. Because the repo is public, I also reviewed what must never be committed: .env, data/, music/, the Lavalink plugin.

Oct 1

Moving to the Raspberry Pi 5: a stumble at every step

I have a Pi 5 (8 GB, running Kali Linux), so I decided to move the bot onto it to run 24/7.

• SSH with a key. Running ssh through a tool with no terminal means I cannot type a password, and it said Permission denied. I created an ed25519 key and copied it to the Pi but was still refused. The -v flag showed the server accepted the key but the client could not sign. The cause: the two quotes "" I typed when creating the key were interpreted by Windows as a real passphrase. I removed it with ssh-keygen -p.
• Docker permissions. Docker was installed but my account was not in the docker group, and sudo needed a password. I split out the parts that need root (installing packages, adding the group) to run myself, and did the rest over SSH.
• Lavalink plugin 404. On the Pi, Lavalink could not download the YouTube plugin because Maven returned 404 for that exact version (it worked on the PC because it had been downloaded earlier). The plugin is a Java file and runs on any architecture, so I copied the .jar straight from the PC. The plugins folder turned out to be owned by root because Docker created it, so I had to change the owner through a temporary container.
• A glued-together line. The .env file had no trailing newline, so the line I appended stuck to ANALYSIS=on and became ANALYSIS=onAUTOSAVE_SECONDS=0. Since then I recheck .env every time I edit it.

Because the Pi uses an SD card (which wears out with heavy writing), I cut disk writes: the "still alive" signal in RAM, AUTOSAVE_SECONDS=0, rotating Docker logs.

Oct 2

Making use of the Pi: tagging, audio analysis, multiple bots

I wanted the Pi to do the things a personal computer could never be left running in the background to do.

• Auto-tagging with audio fingerprints: fpcalc generates a fingerprint and queries AcoustID, and if that fails it searches MusicBrainz by file name. Tags are stored outside the music files (the originals are never modified). In a trial on 10 of my real tracks, AcoustID recognized none of them (they were all rips), so only the file-name search was left, with confidence capped at 80%, below the 85% auto-apply threshold, so they became a queue waiting for my review. I set the threshold that way on purpose so the machine never overwrites tags with a wrong guess.
• Audio analysis to power /vibe (mood radio) and /similar: decode the middle 40 seconds with ffmpeg, then compute energy, brightness and BPM using FFT and autocorrelation, written in plain JavaScript. I tested it with synthetic beats from 80 to 170 BPM. The first version read 140 BPM as 70 (confusing it with half-time); smoothing the onset envelope brought the error under 1 BPM. Running it on the real library, I saw energy "saturating" at 1.0 for loudly mastered tracks, so I measured the actual loudness of the library (roughly −18 to −5 dB) and recalibrated the scale.
• Several bots at once. I added a multi profile to Docker Compose. Since two processes sharing one data folder would overwrite each other's files, I gave each bot its own data and let only one "worker" bot write to the shared folder.
• Operations. Backups with VACUUM INTO (a consistent copy even while the bot is writing; I verified by decompressing and reading it back). A self-update script that pulls from GitHub and rolls back if the bot is not healthy after the update. Uptime Kuma for monitoring. Tailscale for remote access without opening router ports.

Oct 2

The /display status page and the case of the “vanishing title”

I built a small web page that shows what is playing, for phones and big screens, with controls protected by a token. I redesigned it several times: a disc that spins and pulses to the track's real BPM, a page color that changes with each song, a ruler along the bottom made of beat ticks, and tapping the ruler to seek.

When testing with headless Chrome, the screenshots were always missing the title. I spent a while suspecting my code, until I realized Chrome's "virtual time" mode does not run CSS animations, so the text stayed in the hidden state at the start of the animation. I wrote a script that drives Chrome over DevTools, waits for real time, and only then takes the screenshot. Lesson: when a testing tool gives strange results, check the tool before fixing the code.

On the page's security: it is only exposed on 127.0.0.1 on the machine running the bot; controls need DISPLAY_TOKEN; it does not expose Discord IDs or requester names; it hides itself during a music quiz (so as not to give away the answer); and the CSP is strict (no inline scripts, no external resources). The tests also caught a bug: a null value was being treated as 0, so a seek request could send the track back to the start; I fixed it so the API only accepts real numbers.

Oct 2

The 3.5 inch TFT screen: the longest story

I have a 3.5 inch resistive touch TFT that plugs straight onto the Pi's GPIO header. The goal: show what is playing right on the Pi.

1. A white screen

The driver loaded normally and dmesg was clean, but the screen was white. I read the device tree to check the control pins: DC is GPIO24 and RESET is GPIO25, matching the Pi 5's RP1 controller. Filling the framebuffer with solid red was still white.

2. Finding the cause

Looking through the documentation: the vendor only supports the Pi 3B and 4B. The tft35a overlay the Pi was using is not part of the Raspberry Pi kernel but is installed by the vendor's script, and is based on the old fbtft driver. The official piscreen overlay has an extra drm option to use the modern driver. I switched to dtoverlay=piscreen,drm,rotate=90: the screen went from white to black. Black was progress too, because it proved the panel had accepted the initialization commands.

3. Proving the data was flowing

The screen was still black when drawing. Instead of guessing, I read the SPI bus byte counter in sysfs before and after writing one frame: it rose by exactly 307,222 bytes (480×320×2 bytes plus commands). So the Pi was sending everything, and the fault was elsewhere. The next hypothesis was SPI speed: the new driver defaults to 24 MHz, while the vendor designed it for 16 MHz. I lowered it to 8 MHz and the image appeared.

4. Orientation

I drew a test image with four colored squares in the four corners, plus text, to see how many degrees the screen was rotated. The result: it needed a 90° rotation.

5. Touch

My first calibration was wrong: I only recorded 3 of the 4 corner touches, so I guessed the order wrong and mixed up the axes. When I tried the buttons, the touch log showed every press being counted in the bottom-left corner. Cross-checking all the data, I worked out the right answer: no axis swap, just the horizontal axis inverted. Only then did I realize this is a resistive touch screen, so it needs a stylus, and fingers are unreliable.

6. Lag

The first version sent the full frame twice a second (about 614 KB per second, when an 8 MHz bus only handles roughly 1 MB). I tried sending only the changed region and found it was even choppier. Measuring again, drawing took only 12 ms, the bottleneck was the SPI bus, and the spinning disc demanded too much bandwidth. I changed the design: no motion on the small screen, redrawing only once per second (the progress bar advancing) and immediately on touch. Traffic dropped to about 307 KB per second. Lesson: the hardware dictates the design. The same idea is smooth on a phone and not on an SPI screen.

7. The look

I iterated many times by feel: a typographic poster, a big clock when idle, a large avatar, then a magazine cover with a serif typeface. I dropped the border around the photo four times (a colored border, corner marks, a ruler with tick marks, a rectangular frame) until I chose to remove the border completely and let the photo bleed to the edge like a real magazine cover.

Oct 2

Small bugs that ate time

• pkill -f killed my own SSH session, because the string it was looking for was also in the very command being run. How to avoid it: split it into a separate command or use the [p]attern trick.
• The Tailscale service was in the disabled state, so after the Pi rebooted, the control page and Kuma over Tailscale vanished. I now check systemctl is-enabled rather than only is-active, then enabled it at boot.
• git pull on the Pi was refused because I had copied files over by hand earlier. Since then I always commit first and then sync.
• A portrait photo left in the project folder was nearly pushed to the public repo by git add -A; I added it to .gitignore before committing.
• Many long commands in the Windows shell broke on quotes and backslashes; I switched to writing the script to a file and running that.

Oct 2

Cleaning up and rewriting the docs

Finally I cleaned up both machines. On the PC: stopped and removed the project's Docker containers, images and cache (everything had moved to the Pi), and deleted the old data after pulling a fresh backup from the Pi (so a copy exists outside the SD card). On the Pi: removed temp files and leftover images. Things that do not belong to this project I left alone. After that I split the website into several pages to make it easier to read and wrote this devlog.

Oct 2

Version 2.4: playlists, radio, stackable filters and several nodes

After the cleanup I went back to the bot. This batch collects the things I missed while using it every day.

• Audio filters. /filter used to allow one effect at a time. It now has subcommands: effect toggles an effect on or off (bass boost, nightcore, vaporwave, 8D, karaoke, tremolo, vibrato, mono) and effects stack, speed and pitch from 0.5 to 2x (changing one leaves the other alone), eq for bass, mid and treble from -2 to 10, plus status and reset.
• Playlists. /playlist comes in two kinds: personal (yours, working on every server) and server playlists (shared; if the server sets a DJ role, changing them needs that role). They hold both library files and online tracks, up to 25 playlists per owner and 200 tracks each. /privacy delete removes personal playlists.
• Server radio. /radio plays endlessly from the music library. Each round is ordered by the server's 👍/👎, the tracks it plays through versus skips early (last 90 days), and what it usually plays around the current hour; with ANALYSIS=on it adds a small push toward a mood that fits the time of day (chill at night, steady in the morning, upbeat by day). The radio survives restarts.
• Wrapped and stats. /wrapped and /mystats gained format (card, or a 9:16 1080x1920 story for stories), theme (dark or light) and private (only you see it).
• Quiz. /quiz start gained mode: guess the song (default), the artist, the release year (exact is full points, within 2 years is partial, and it needs a year tag) or from a lyric line. /quiz top gained period: this month (default), last month, all time. Quiz seasons are calendar months. Two new badges, Bat Ears (50 correct answers) and Monthly Menace (top quiz player of a finished month), bring the total to 10.
• Several Lavalink nodes and sharding. The LAVALINK_NODES variable lists several nodes, for example main=lavalink:2333,backup=lavalink2:2333; when one node goes down, its players move to another. The second node starts with docker compose --profile ha up -d, and every node must see the same music folder. Sharding is optional (SHARDS=auto or a number); all shards run in one process, and one shard is enough up to 2,500 servers. /stats now shows nodes up and the shard count.
• One-command installer. scripts/install-pi.sh installs Docker and git, asks for the bot token and application ID, generates the passwords, starts the bot, registers the commands, and can optionally install the backup and auto-update timers and set up the TFT screen (--tft). It is safe to run again, and --dry-run previews it.

The bot now has 38 commands. I also rewrote the privacy statement on the website to be plainer: the bot is self-hosted, the data lives in data/ on the bot's own machine, and it does not read message content.

Oct 2

Companion bots, and why they left

A server with a music bot can still feel empty between songs, so I built a set of small chatty bots: five personalities that ask questions, tell jokes and riddles and share fun facts, with another bot jumping in when nobody answers and everybody stepping back when a person joins. They stay clearly bots (BOT tag, no pretending to be human), keep quiet hours and a daily limit, and read no message content.

I first put them inside this repo as src/companions/, then moved them out the same day. They share nothing with the music bot at runtime: no Lavalink, no database, no music config. Keeping them here would have meant that anyone who wants lively chat downloads a music bot, and that an update to one restarts the other. So they became their own project, companionsDISCORD, with their own repo, Docker image, auto-update timer and website. On the Pi they are two folders and two compose projects; the container names, image tags and data folders differ, and the companions publish no ports, so nothing clashes.

• Lesson: when a feature has its own process, its own data file and imports nothing from the rest, it is already a separate project; the repo split is just paperwork.

What I learned

• When a direction depends on something you do not control (YouTube), change course early.
• Measure before optimizing: the SPI bus byte counter, draw time, traffic. Guessing wrong costs more time than measuring.
• Break hardware problems into testable hypotheses: which pin, is the data flowing, the speed, the rotation.
• Data from outside always has to be sanitized before it is displayed or written to disk.
• The design has to fit within the hardware's limits, not the other way around.
• For a feature that could cause harm (downloading copyrighted music), weigh the risk and choose the safer option instead of forcing it through.
• Operations matter as much as writing code: backups, self-updates with rollback, monitoring, auto restart.
• Write tests for the parts with real logic (BPM, path blocking, the API, string sanitizing) and be clear about what has no tests.

Still unfinished

• I have not tried SPI at 16 MHz (it might allow motion on the TFT screen again).
• I have not verified that the TFT screen comes back on its own after the Pi reboots: the @reboot line in the crontab has not been through a real test.
• The TFT Python code has no automated tests; for now it is only checked with exported images and by running on the device.
• The rollback part of the self-update script has only been written and tested for the "no new version" case; I have not tried it with a genuinely broken release.
• The multi-bot profile (multi) has not been run with a second token.
• The SD card is still the weak spot; for long-term use, move to a USB SSD.