Node.js was one of the earliest technologies whose rise I got to witness as a part of the tech community. Everything was new (Can you imagine a world before npm?), and everything was evolving at a rapid pace.
But where there is high speed there is friction.
And so the drama unfolded around Node.js, its stewardship, technical decision-making, and gatekeeping. Of course, in hindsight we feel like everything was destined to succeed - but back then, things weren’t always so clear and even less straightforward. “Node.js - The Documentary” does a great job of offering perspectives of the people involved and letting them tell their stories - without drama, yet you can still feel their excitement, engagement, and connection to the project.
It’s easy to recommend this. Go watch it.
]]>Given the lazy web developer that I am, these videos were of course created in HTML/CSS (with a pinch of TypeScript) because when all you have is a hammer, every problem looks like a nail. Plus, I wanted to try out a few new things.
I figured, this might be of interest, so let me pull back the curtain on how we built this:
Given a podcast episode’s mp3 file, I would like to …
timestampA
to timestampB
Because I really just wanted to prototype something quickly, I chose Deno as the programming language to tie this all together. Deno is great - in my humble opinion it is better than NodeJs because it takes care of a lot of the build process and process isolation and provides default storage, queues, deployments, etc - while still using TypeScript.
I also chose to make this a small CLI tool so that I would not have to care much about design - (which I am not good at, to say the least). Hence I went with the commander package.
Providing all the metadata via the CLI is pretty straightforward. I use a mix of select (basically dropdowns) and input (basically text fields) prompts that are all handled by the cliffy Deno package. (In fact, it brings a lot more capabilities to the table, for now, I only needed the user prompting).
All of the metadata is then stored in Deno’s native key/value store.
There are some packages available on npm that offer mp3-cutting functionality. However, many of these simply use
ffmpeg
under the hood. So I decided to forgo another dependency and use ffmpeg
directly.
The Deno.Command
API makes it pretty straightforward to create a child process.
And then all you need to do is, run the right ffmpeg
command - such as ffmpeg -i file.mp3 -ss 10 -to 20 output.mp3
-
to create a 10-second clip.
For the programmier.bar website we already experimented with several speech-to-text APIs to generate our podcast transcripts.
For now, my recommendation is to simply go with OpenAI’s Whisper,
as it has the best pricing/performance ratio (though your mileage may vary).
The API usage is pretty straightforward, we simply post our spliced mp3 file via fetch
and receive the transcript as a response.
Two limitations are worth pointing out here:
Now, what is a video? A video is the combination of an audio track with a sequence of images or “frames”. We have already taken care of our audio track - but what about the frames? For the next step, we need to generate the frames. And at a constant 30 frames per second, we need quite a lot of them.
The initial idea was to use HTML and CSS and simply “render” the output as an image. Libraries like satori allow us to render HTML straight to PNG - and are blazing fast too, because no browser, requests, etc is involved. This - together with some caching - allowed me to generate the hundreds or thousands of frames required in a few seconds.
To streamline the HTML generation I decided to use a templating engine. Ideally, I would want to use twig (because I know it by heart) or use some of the technology underlying one of Deno’s many web frameworks. Given that this was a prototype, however, I decided to go ahead and simply use Eta - it is lightweight but powerful enough for our use case - and most importantly, battle-proven and well-documented.
But (and you knew there was a but coming), given that these packages have to implement (at least parts) of a browser rendering engine, there is always something off in their rendering - or you start facing odd limitations (e.g. satori can only render from JSX and requires the use of flexbox for everything).
So eventually, I decided to abandon the “straight from HTML/string to image” approach in favor of something more traditional.
(Nevertheless, kudos to satori for accomplishing this feat - and I can imagine that for smaller rendering tasks this is the best way to go).
Using puppeteer, we now use the generated HTML, load it into an actual (headless) browser, and make a screenshot
(For this, I would recommend, using PNG as an output format for less compression artifacts and a deviceScaleFactor
of 2).
Even with the same caching smarts added, this approach takes an order of magnitude longer to generate all the frames. It is not quite real-time, but almost - taking roughly ~45 seconds for a 60-second clip.
Finally, we need to merge all the frames and the audio track into one video file. For this, again, we use ffmpeg
and the Deno.Command
API:
ffmpeg -framerate 30 -i output_%04d.png -i audio.mp3 -c:v libx264 -crf 15 -c:a aac -strict experimental -b:a 192k -shortest video.mp4
.
Et voila, we have our finished video, ready to be shared across social media.
Now, so much about all the “fun” part. The question remains: is it worth it? Absolutely.
Even in our limited testing, we see that our posts that include video have quite a few impressions more than our traditional posts using static images or only text. And it makes sense, even today, video isn’t ubiquitous in many social networks (at least not the ones, we are most active on. If you’re a professional TikToker, you might disagree). So it is only natural for these posts to draw more attention, making this differentiation invaluable.
There is more to it, though. We had experimented with video before, but these videos were always created but our very talented colleagues in the video department. Their time is limited and their schedule is not always flexible. Now, however, this tool allows us to produce videos at almost no cost, and we can even adjust the graphics ourselves because it is all just HTML and CSS under the hood.
This solution is not only effective and empowering but also economical.
Thank you for your time! I hope you’ve found this overview enlightening, whether it introduced you to Deno’s capabilities or simply entertained you.
P.S.: Interested in learning more or having me discuss this project at your event? Don’t hesitate to get in touch!
]]>“You can’t connect the dots looking forward…” Steve Jobs
First and foremost, I was swooped out of Shopify with the second big wave of terminations in May. Of course, I had been between jobs before. But never - since high school - was I out of a job for longer than a month. This summer marked the first time of me being unemployed for several months.
It felt very weird. I had always been working, always traveling and even when I was home, my mind was constantly spinning around that one problem that still needed a fix.
Certainly, there was a lot to learn in that period - about me (maybe more on that in a later blog post) about employment law (certainly more on that in a later blog post), and about the job market and my place in it (probably out of date by the time I’d get to writing about it).
“… you can only connect them looking backwards. …” Steve Jobs, continued
But, as so many times in my life before, I was very fortunate and found a new home at Lotum. Here I get to work with great people and meet even more people from all over the tech landscape - not to mention getting to host a podcast- and meetup series. In short, all the stuff I loved as parts of my previous jobs, but now I get to do them almost exclusivly.
I have always loved this speech by Steve Jobs, but never before in my life has it hit closer to home than this year.
Hence, let us close with:
]]>“… So you have to trust that the dots will somehow connect in your future. You have to trust in something - your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all the difference in my life.” Steve Jobs, continued
Recently, an old friend asked me to assess a company for a potential investment on their part. The details of the (potential) investment and the actual company aren’t crucial for this post, so I’ll refrain from disclosing further information.
What caught my attention during my “research” was the company’s apparent aversion to deploying on Fridays. They claimed they made this decision for the sake of their employees’ peace of mind. At first glance, this seems like a commendable choice. However, I’m here to share why, in this author’s humble opinion, they may be misguided, albeit with good intentions.
I have no doubt everyone at that company is acting in good faith. But it seems that they might not have thought this through all the way to the end. While reducing Friday deployments might alleviate short-term stress, it also perpetuates the myth that deployments, in general, are ominous and warrant respect or fear. This, in turn, may elevate stress levels during deployments, even if they happen on other days of the week.
Now, here’s where the purists might chime in and argue:
You should never be afraid to deploy!
If you are afraid to deploy, then your deployment process is flawed!
I strongly believe that this is a false dichotomy. It is not a question of if if your deployment will ever fail (and potentially in a disastrous way), but rather of when that will happen. No amount of processes, automation, or quality assurance can ever reasonably provide 100% security. It is always a trade-off and one with ever-diminishing returns.
While I’m a proponent of robust CI/CD pipelines, I implore you not to fall into the rabbit hole of endlessly adding layers in an attempt to buy more security.
Instead, consider a simpler way to reduce deployment stress: easy rollbacks! Every action and decision becomes easier as the cost of reversal declines. (This is also a great way to measure “good” software architecture, but that’s another story.) If rolling back an erroneous deployment requires just one click, fear diminishes. No more panic scrolling through the code to find the cause of a bug discovered on the production system. Instead, you simply roll back the deployment, take a step back, and calmly analyze the code.
(Of course, there are even more things to be done here: Partial rollouts, solid telemetry, robust backups, and so on. But eventually, you will end up with the same fringe benefits as in the deployment automation. So be aware of the balance you need to strike here).
Our world is complex, and we seldom deal in absolutes. Reality is never just black or white and always consists of a hundred shades of grey.
So, please keep in mind that there may still be reasons to be cautious during deployments (huge database migration anyone? re-platforming recently?).
What I want you to take away from this, is that if you think you have a problem with your deployment strategy the solution probably isn’t to just add more. Instead, we should always be looking for solutions in other areas as well.
]]>Some of my co-hosts suggested that there might not be many holdovers from the current job of a „programmer”. Because in the (potentially near) future much, if not most, code would be written by large language models (LLMs) and all that would be left for the “programmer” to do was to prompt them properly. In fairness, the prevailing sentiment in the industry aligns with this perspective, foreseeing a drastic transformation or potential disappearance of the profession.
Here’s where I ardently disagree.
I firmly believe that our profession isn’t defined solely by the code we write, let alone the current methods we employ. If that were the case, proponents of Assembly might argue that those using higher-level languages shouldn’t be labeled “programmers.” The same argument could extend to individuals who wrote code on punch cards or threads, suggesting that these digital natives might not comprehend what “real” programming entails.
Programming inherently tends to become more abstract and higher level over time. (When was the last time you had to worry about a pointer in memory?) Some languages even explicitly mimic human language for readability (Objective-C, anyone?). Yet, we don’t withhold the “programmer” label in these cases.
So, when people use human language to prompt LLMs to generate code and software, should we perceive them as less skilled? I might be going out on a limb here, but there might also be a very primitive fear at play here: the fear of becoming obsolete. Hence, the notion of framing our current way of working as a form of higher art that is very different from what future “programmers” would be doing. (Ada Lovelace is probably turning in her grave right now…)
If writing code isn’t the essence of being a “programmer,” then what is?
First of all, I suggest the term “software developer” over “programmer”. I know this is a very blurry line and everyone has their own definitions here, but in my book, “programming” simply implies typing out code whereas “software development” implies a more holistic approach to the craft. Your interpretation may vary.
Regardless of terminology, the essence of software development is not merely to produce code. The goal of software development is to turn (business) requirements into software so that it can deliver value. That requirement can be very personal to you, or be part of a business; the software can be a complex application or a small script; and the value can be anything between making money, scratching an intellectual itch, or just providing fun and leisure. In that process, coding is one tool (of many) to get the job done, but it is not a goal in and of itself.
Are you working on an operating system? Congratulations, you’re a software developer! Automating a home task with a small script? Congratulations, you’re a software developer! Manipulating data programmatically in data science? Congratulations, you’re a software developer! Building something with no-code tools? Congratulations, you’re a software developer!
So what I want you to take away from this is: that learning and knowing how to write code is not what makes you a software developer. It might be a fantastic starting point - but there is so much more. Which is why getting to write less code does not necessarily make you any less of a software developer. Understanding business requirements, anticipating edge cases, finding creative solutions, foreseeing future needs and requirements, learning from user feedback and learning when to (not) ignore it, learning about architecture and patters, the human element in collaboration and so, so much more - enough to write a lifetime worth of books on the subject.
And who knows, maybe … in the distant future … AI could be universal enough to handle all these tasks. But even then, someone is still going to have to interact with that AI in some form or manner, and that someone … is going to be a software developer.
Afterword
This discussion could easily be dismissed as “being about semantics only”. But I strongly believe that language matters. In using certain language we have a choice between elitism and inclusivity. Even if we seldom discuss these matters, language molds our mental model of the world, making these discussions crucial.
I’d rather live in a world that embraces change and welcomes more people into our craft than one that raises barriers to entry.
]]>Lightning talks are usually shorter talks (10 to 15 minutes, sometimes even less) with fewer or even no slides to accompany them. And while the concept in general is not new (Wikipedia dates the first lightning talks in the late 90s early 2000s), last week was the first time we applied the concept to a programmier.bar meetup.
In some communities lightning talks might be frowned upon, because they are often perceived as the equivalent of „an open mic“ at a meetup or conference - especially at the latter where people probably paid a ticket-fee and in turn (fairly) expect high quality content.
In my humble opinion that take is a bit short-sighted. I like to think of lightning talks as lowering the bar, while at the same time raising it. Let me explain that paradox:
Lightning talks are shorter and require less (or even no) slides, code demos, or any other form of supportive content. Thus they are lowering the barrier of entry. And that is intended.
It makes it feel less overwhelming and daunting for first time speakers to take that first step and get out there. But don’t be fooled. The same applies for experienced speakers (including myself here). Just because we have given long talks and presentations before doesn’t mean we necessarily want to go through that process all the time and for every topic. We too appreciate the chance of not having to prepare a huge slide deck just to be able to talk about a subject that might be near and dear to us.
Then of course there is the other side:
Lightning talks are shorter and allow for less (or even no) slides, code demos, or any other form of supportive content. Thus they are raising the barrier of entry. And that is intended.
Talking about a complex subject at length is easy. But breaking it down into a short, concise and capturing talk - without skipping on the important and necessary details - is not. It is not for nothing that famous French philosopher Pascal already noted in the 17th century that “I would have written a shorter letter, but I did not have the time“1. Hence, doing so well can be a challenge even for an experienced speaker.
So what I would like you to take away from this is: Whenever you see lightning talks listed on an agenda somewhere, don’t dismiss them right away. Instead take them for what they are - a format that can be challenging for all kinds of speakers. And a format that can be entertaining and educational - if only you let it.
This quote in fact has been attributed to many great minds, but Pascal was the one I found most often. So take that with a grain of salt. ↩
In that episode, I mentioned that while I appreciate the Ruby on Rails documentary as a whole - and most certainly love the fact that this kind of content is produced in the first place - this one in particular stood out for me.
The documentary prominently features David Heinemeier Hansson, co-founder of Basecamp and the creative/technical force behind Ruby on Rails. It’s an indisputable fact that Rails owes much of its success to DHH (as he often nicknames himself). In the documentary, Tobi Lütke (of Shopify fame) makes that point particularly clear.
But here lies the crux of the matter. DHH, in the past, has made statements and decisions that, IMHO, can be perceived as controversial. From implementing policies at Basecamp that curtail discussions on societal and political issues among employees1, to expressing skepticism about the principles of “Diversity, Equity, and Inclusion”2, and even making technical decisions like unilaterally removing TypeScript from the Turbo library3, his approach has generated significant debate.
While everyone is entitled to their opinions, especially if they’re at the helm of a company (insert with-great-power-comes-great-responsebility meme here…), the documentary’s focus on Ruby on Rails, inexorably linked with DHH, opens a Pandora’s box. Honeypot, in showcasing the Ruby on Rails story, missed an opportunity to provide its audience with a more nuanced understanding of the person behind the framework.
Let me be clear—I’m not attempting to cast a shadow on Ruby on Rails or Honeypot. Nor am I attempting to answer the (age-old) question of whether one can separate the art from the artist (everyone needs to make up their own mind about that). My point merely is that Honeypot could have improved the audience’s experience by offering some context on the individual they chose to highlight in their documentary.
In a world where technology (and thus the individuals behind it) play an increasingly influential role, a more holistic understanding of the personalities shaping these and technologies (in this case frameworks) becomes ever more important. As developers and/or users of these technologies, understanding the context surrounding the creators can lead to a more informed and nuanced perspective.
]]>$your_favorite_deity
for coderetreats!
In the latest episode of the programmier.bar podcast I had the opportunity to speak about code retreats and why I think they are a fantastic tool to level up your game. Hence, I thought I should take some time and elaborate in more detail here.
So, what’s a coderetreat, anyway? Well, think of it as a coding dojo where you can kick back, code, and perfect your skills. These events usually span a day and revolve around cracking coding challenges - often times Conway’s Game of Life is choosen because you can basically implement it in various levels of complexity but still get fast results and thus can iterate quickly. The day is divided into rounds, each focusing on different coding aspects. It’s like a video game with levels, but instead of slaying monsters using ever changing gear, you’re slaying a coding problem with ever changing constraints.
Coderetreats are the perfect playground for honing your coding skills. They provide a platform to practice problem-solving and delve into algorithmic thinking. And they really remove all the excuses to not be embracing test-driven development (TDD).
In coderetreats, collaboration is the name of the game. You’ll find yourself working closely with other developers, which enhances your communication and teamwork skills. Effective communication is often more critical than pure coding prowess - that’s a hill I’m willing to die on.
One remarkable aspect of coderetreats is the opportunity to explore new technologies. It’s a risk-free environment to expand your tech stack beyond the confines of “real-life” projects. Learning to adapt to different languages, styles, libraries and frameworks is a valuable skill every developer should master - remember: only bad crafters blame their tools!
Most people love to experiment - and many coders are curious by nature. And coderetreats are the ultimate lab. Participants can embrace creative problem-solving by challenging themselves. Whether it’s avoiding if-statements, loops, or primitive data types, or writing only stateless code, coderetreats are your place to push the boundaries. Plus, there are different types of pair programming to spice up the experimentation.
If you want to find out more on the topic, I recommend you check out coderetreat.org! And also, don’t forget that every November there is the Global Day of Coderetreat. The opportunity to find virtual or in-person events (often hosted by generous companies in a meetup-like fashion) to connect with like-minded people and enjoy some coding together. For a list of events check out the website.
You don’t need much to get started. Make sure to have your laptop, charger, and coding environment ready. Mental preparation involves an open mindset and a willingness to embrace new challenges … and have fun!
I attended my very first coderetreat in 2017 at the SoCraTes conference - it was a wonderful experience and I would like to thank the organizers, facilitators and of course the fellow attendees for making it so. It is one of the reasons I have kept coming back to that event every year since then.
]]>Naturally, some of the podcasts that I listen to are better than others. Some even stand the test of time, delivering timeless content that might very well be worth a re-visit in five or ten years time. That might become a problem though. Like all content on the internet, there is no guarantee that this podcast’s feed or file hosting might be around that far out in the future. After all, continuous feed and file hosting costs money and no-one wants to foot that bill for all eternity.
Luckily, local offline storage becomes ever cheaper, enabling us to simply keep a local copy of our favorite podcasts. But how to go about downloading your favorite podcast? Of course, one could simply use any podcast client (back in my day, these were called podcatchers) and then find the files it has downloaded - probably somewhere deep in your OS’s library folders. But even then, you would still need to open that podcast client, add your feed, mark all the episodes for download and wait…
It very much feels like there must be a better solution to this. And luckily, there is: podcast-dl by Joshua Pohl is a handy CLI tool to download whole podcast feeds.
There is only one problem though: If you have multiple podcasts you continuously want to keep an up-to-date local copy of then things turn tedious very quickly. Because you end up having to run that command on a schedule, for every podcast.
Which is where my small addition comes in: https://gitlab.com/JanGregor/podcasts-archiver is a dockerized version of Joshua’s great tool that comes with a twist: Now, you simply need a folder structure like:
podcasts/
├─ podcast-a/
│ ├─ URL
├─ podcast-b/
│ ├─ URL
│ ├─ OPTIONS
And simply point the docker container to that folder structure like: $ docker run -it -v
pwd/podcasts:/podcasts registry.gitlab.com/jangregor/podcasts-archiver /podcasts
and let the magic happen.
The included script will automatically go through every (1st level) subfolder and try to find a URL
file in there that keeps the podcast’s URL - if an OPTIONS
file is also present, it will use its content to append the command. And then it will simply run podcast-dl
for every of your podcasts/subfolders.
Now, do you really need a container for that? Could you not do with individual CRON jobs? Maybe, probably even, but for my particular setup (running this on a schedule on my NAS to permanently keep a copy there) this was the easiest and most portable solution. Hence, I decided that, if I already built it in a portable way, I might as well share it with all of you out there.
]]>Defensive programming is a form of defensive design intended to develop programs that are capable of detecting potential security abnormalities and make predetermined responses. It ensures the continuing function of a piece of software under unforeseen circumstances. Defensive programming practices are often used where high availability, safety, or security is needed.
Or, if comic style comparisons are what speak to you
Defensive programming? That’s your Jarvis, your AI companion, constantly whispering in your ear, “Sir/Ma’am, potential issue detected.” It’s like you’re in the midst of a high-stakes battle, but instead of lasers and missiles, your weapons are meticulously crafted if-else statements and boundary checks.
Either way, there are three sources of errors to guard against
In an interview with NPR Suzanne Dodd, project manager for the Voyager Interstellar mission, explained:
Well, it was a bit unfortunate. We sent a command to update its pointing toward the Earth, and there was an error in that command. And so it’s pointed about two degrees off of the Earth. And from the distance that Voyager 2 is, close to 13 billion miles from us, that essentially points it almost to the orbit of Jupiter.
So it seems like Voyager 2 is faced with a #2 error. It happens. Even to the best of us. I’m sure we have all been there, crashing a production environment here, dropping a live database there. You learn from your mistakes, no shame, no blame, no hurt feelings.
But then for most of us the stakes are not as high as potentially sabotaging a spacecraft that is a ridiculous 20.000.000.000 kilometers away from earth, with no hope of reaching the craft to patch your code if something goes wrong and you loose contact.
But turns out these NASA engineers are some smart cookies! It is rocket science after all, huh? As published by NASA’s Jet Propulsion Laboratory (JPL):
Voyager 2 is programmed to reset its orientation multiple times each year to keep its antenna pointing at Earth; the next reset will occur on Oct. 15, which should enable communication to resume. The mission team expects Voyager 2 to remain on its planned trajectory during the quiet period.
That, my friends is some solid defensive programming right there! If unforeseen circumstances should occur, don’t panic, but rather try to continue as planned if you can. And in the meantime reset any parameter that might stem from bad user input (in this case the antenna orientation) back to sensible defaults.
It might also be worth mentioning that Voyager 2 was launched in August of 1977 - 46 years ago! Plus, it has its conceptual roots in the “Grand Tour” program which was first conceived in the mid 1960s - a decade before!
So, if defensive programming was feasible for engineers half a century ago, I think it is fair to say we should all challenges ourselves a bit more often and take a page out of NASA’s playbook here. After all, Murphy is waiting for all of us…
UPDATE: It turns out, while the self-recovery procedure is a genious solution, it was necessary after all. NASA’s Deep Space Network (DSN) has apparently become strong enough to simply “blast” a signal into the general direction of Voyager 2 and re-establish contact. But … let that not take away from the learnings above!
]]>