Adventures in Software Rot: Avoiding Getting Stuck
Over the years, I’ve often discovered open-source abandonware — to my chagrin, often it was software that I needed at the time. Sometimes I’d choose to dive down the rabbit hole, seeing what exactly it would take to get it running again.
My usual strategy is to try to find the original dependencies. In the case of one adventure with a Node.js app, I discovered that old versions of things within the Node.js ecosystem, including Node.js itself if I remember right, can be difficult to find (not-so-dodgy) sources for. Short of getting into recompiling ancient Node.js binaries myself, which was outside the scope of what I was willing to muck with, I soon hit a wall and had to try upgrading various JS dependencies.
nvmmakes it easy to install older versions of Node.js, and even if a few dependencies needed upgrading, Node.js had a great reputation for being backward-compatible.Unfortunately, bitrot won. An obstinate approach and Node.js back-end compatibility were no match for endless deprecations and impossible dependency loops among old versions of Grunt, Bower, and dozens of other components. Hours later, it was beyond clear that it would be much easier to start from scratch—this author’s own advice against reinventing the wheel notwithstanding.
With an entirely different stack, getting the Jekyll-based blog you’re currently reading going again after its initial drafts in 2017 also took a lot more troubleshooting than expected.
It’s with such adventures in mind that I’m a bit trepidatious when I think of checking into why I can’t run Willy and Mathilda’s Houseboat Adventure: The Game on my more recent Linux Mint machines anymore. (Thankfully, its Windows version runs fine on Linux using Proton. But not everyone runs Proton.) I didn’t even develop it on Linux, originally; at the time, once the game was finished, I used Linux VMs (Ubuntu, I believe) to target 32-bit and 64-bit Linux.
With yet another stack at play, one based around Haxe and HaxeFlixel, I have no idea whether it’s possible to stick with the original versions and just debug library dependency issues (already not my strong point) or if I would be forced to update the stack and therefore probably the source as well.
–
Since I had first developed it for Windows, I had HaxeDevelop files lying around, but I’m pretty sure I didn’t (and couldn’t) use these on Linux. Thankfully, I had created a small environment setup script for my VMs, because I didn’t have a lot of spare space in those days to keep VMs around, so I had to make the setup reproducible, not just as a matter of best practice:
haxelib set openfl 3.6.1
haxelib git crashdumper https://github.com/IBwWG/crashdumper
haxelib install systools
haxelib install random
haxelib git flixel https://github.com/codingthat/flixel https://github.com/codingthat/flixel/tree/all-my-prs-are-belong-to-me
pushd /usr/lib/haxe/lib/flixel/git/ ; git checkout all-my-prs-are-belong-to-me ; popd
openfl test linux
(Hmm…I’d better add some version numbers to systools and random. And check on that all-my-prs... business. Back to that in a moment.)
OK, looks like I relied on Haxe and haxelib already being installed, and relied on my custom forks of a few projects. Those will at least be frozen in time, but which Haxe had I used? Interestingly, neither my .hxproj nor Project.xml helped here, though they held other library version numbers, e.g., for systools. Maybe Git history? My own devblog? GitHub issues I had reported around the time?
Something like that. OK, is Haxe 3.4.0 still available?
Not far off, 3.4.2 is listed, but…only for Ubuntu 16.10. I happen to be using Linux Mint 20.3, which is based on Ubuntu 20.04. Hence apt list --all-versions haxe gives me just two options:
haxe/focal 1:4.3.4-1~bpo20.04.1 amd64
haxe/focal 1:4.0.5-1 amd64
There’s my first dilemma. Should I chance my environment by installing the older version anyway? Or get a Docker container with Ubuntu 16.10 running for this? Maybe a full VM? Or see if my code still works with Haxe 4.0.5? I don’t happen to have Docker up and running on this machine at the moment, so the path of hopefully least resistance is trying my old code against a much newer compiler. Here goes, with sudo apt install haxe=1:4.0.5-1…
After that, running through an updated enviornment setup mostly worked, except that my fork of crashdumper pulled with it the newest OpenFL despite the explicit version set:
haxelib set openfl 3.6.1
haxelib git crashdumper https://github.com/IBwWG/crashdumper
haxelib install systools 1.1.0
haxelib install random 1.4.1
haxelib git flixel git@github.com:codingthat/flixel.git all-my-prs-are-belong-to-me
That was an easy fix, though: crashdumper had "openfl": "" in its haxelib.json, which, the Haxelib docs say, means it will install the latest version. Just needed to force it to 3.6.1 to match my setup. Along the way, I noticed that there was now an easier syntax for haxelib git to pull a specific Git branch, so I made use of it too. I noticed further in the docs that I could put all these dependencies into a new .hxml file (my preexisting .hxml seemed to be Windows-specific) and just haxelib install mynew.hxml, but the above commands allows me to debug line-by-line if anything were to go wrong.
Which it did.
After the remaining lines, OpenFL is installed, but the command openfl isn’t, because I only installed it via haxelib, but didn’t set it up. The fun begins: haxelib run openfl setup yields Error: Failed with error: No such Project : lime:2.9.1 (2.9.1 being the version I need, that I used at the time).
I guess I need haxelib run lime setup first. That worked, because I could then:
$ lime --version
___
/\_ \ __
\//\ \ /\_\ ___ ___ __
\ \ \ \/\ \ /' __` __`\ /'__`\
\_\ \_\ \ \/\ \/\ \/\ \/\ __/
/\____\\ \_\ \_\ \_\ \_\ \____\
\/____/ \/_/\/_/\/_/\/_/\/____/
Lime Command-Line Tools (2.9.1)
Except that haxelib run openfl setup still fails with the same error. haxelib set lime 2.9.1? No, no change. Same with updating haxelib itself (to 4.1.0).
Due to my special talents, the error isn’t Googleable (though this post should change that!) but without lime mentioned, I gathered that this is an error from haxelib, not necessarily specific to openfl setup. But Googling "haxelib run openfl setup" "No such Project" gave only one result, and not about this particular problem.
OK, let’s dig into openfl setup. That’s a command given to haxelib run, and its docs say I can specify a version there. Let’s try haxelib run openfl:3.6.1 setup. Nope. haxelib remove openfl 9.3.3? Nope, even though 3.6.1 is the only remaining version according to haxelib list. haxelib --debug run openfl setup changes nothing, either.
Thankfully, it’s open source, so what’s in OpenFL’s haxelib.json? Nothing, so run.n, which is complied from RunScript.hx. But all it seems to do is install Lime if it isn’t already, set the current working directory, then… Sys.exit (Sys.command ("haxelib", args.concat (["-openfl"])));. But maybe it’s failing before that, at new Haxelib ("lime")…
record scratch
Turns out (from a different route of trying things like lime test neko and Googling the resulting Cairo.hx:34: characters 13-21 : Unexpected operator) that I don’t only need to think of how my code may not work with a newer Haxe version — Lime itself had to change between Haxe 3 and Haxe 4. The earliest Lime with that particular fix is 7.1.0, so, time to try haxelib install lime 7.1.0. Meanwhile, lime setup linux -64 with Lime 2.9.1 tries to install a bunch of 32-bit stuff, which I don’t particularly need or want, and have had a bad experience with in the past on Linux Mint 64-bit. So if I can dodge that but have a working lime test neko and lime test linux, I should be one step closer. Of how many steps, unclear.
It’s promising: With Lime 7.1.0, lime test neko gives three warnings about typedef deprecations in ThreadPool.hx, but produces a lime create project blah test window no problem. Same warnings with lime test linux, but indeed, it compiles and creates a native window.
And back to our main blocker…here’s what happened (minus some download meters and redundant install messages):
$ haxelib run openfl setup
You already have lime version 2.9.1 installed
Set lime to version 2.9.1 [y/n/a] ? n
hxcpp is up to date
lime-samples is up to date
Downloading openfl-samples-8,7,0.zip...
Downloading actuate-1,9,0.zip...
Downloading box2d-1,2,3.zip...
Downloading layout-1,2,1.zip...
Downloading swf-3,2,0.zip...
Do you want to install the "openfl" command? [y/n/a] ? y
$
Yes! Oddly, trying openfl test blahplatform shows it’s still refering to Lime 2.9.1. haxelib remove lime 2.9.1? No, now it just complains, Error: Could not find haxelib "lime" version "2.9.1", does it need to be installed?. Where can I find which version OpenFL is trying to use? Oddly, it’s not where I would expect, especially given a fundamental dependency. Grepping locally where Haxelib installed OpenFL, I found it in include.xml, but oddly enough, the 3.6.1 tagged repo differs on that point. OK, so I’ll cut those in the local version, for the moment.
$ openfl test neko
~/haxelib/openfl/3,6,1/openfl/display/Stage.hx:10: characters 8-41 : Type not found : lime.graphics.CanvasRenderContext
Ah, yes. This has moved since Lime 2.9.1. OK, which OpenFL goes with Lime 7.1.0? 8.5.0? That’s quite a jump. Even then, not working with this Haxe version:
$ openfl test neko
~/haxelib/openfl/8,5,0/src/openfl/display/IGraphicsData.hx:11: lines 11-23 : @:fakeEnum enums is no longer supported in Haxe 4, use extern enum abstract instead
Oddly difficult to find commits about this, but it looks like 8.9.0 was the first OpenFL to fix this file. But actually it was just moved elsewhere. 8.9.1, then? If so: Warning: Lime version 7.1.0 may not be compatible with OpenFL 8.9.1 (expected version 7.5.*).
Well, with 7.5.0, I get a new slew of errors.
And I’m sure the further I get from the original versions, the more likely it is that HaxeFlixel won’t work with them. And upgrading that, I’m pretty confident will cause breaking changes in my own code.
I’ve clearly gone too far down this road already. Back to trying an older Haxe.
Oh… apparently I should’ve tried lix from the start.
lix helps you avoid “software erosion”, so that when you come back to a project 6 months later on a different computer, you can still get things compiling exactly as they did last time you were working on it.
If only I had Googled in a wider way: “how to install an older version of haxe” rather than “linux apt install an older version of haxe.”
But there are also concerns with this route. Nonetheless, let’s try.
Lix to the rescue?
Well, it seemed promising. But Lix hardcodes a download of Neko 2.3.0.
Also I’m not sure if it installed both that and Haxe 4.3.5 before I had a chance, or if they’re for some other reason already living in ~/.nvm/versions/node/v16.13.1/bin/, but which traces them to there even after I apt remove both Haxe and Neko.
Yep, Lix put them there.
How can I get back to 2.1.0? And will that actually fix the errors I’m getting trying to run lime -version now?
I can just unpack it into ~/haxe/neko. And it doesn’t fix the errors. Nor does lix lime -version do anything different:
$ lix run lime -version
Called from ? line 1
Called from CommandLineTools.hx line 1400
Called from CommandLineTools.hx line 25
Called from a C function
Called from CommandLineTools.hx line 64
Called from CommandLineTools.hx line 1097
Called from /usr/lib/haxe/std/neko/_std/sys/io/File.hx line 30
Uncaught exception - [file_contents,/haxelib.json]
Failed to invoke `haxelib run-dir lime "~/haxe/haxe_libraries/lime/2.9.1/haxelib" -version` because Error: Command failed: haxelib run-dir lime "~/haxe/haxe_libraries/lime/2.9.1/haxelib" -version
Haxe Manager to the rescue, then?
The fourth result for “how to install an older version of haxe” on DuckDuckGo was Haxe Manager:
Easily download and switch haxe versions (currently tested on
ubuntu-latest,macos-latestandwindows-latestvia github actions).
The result:
$ haxelib run lime setup
A new version (4.1.0) of haxelib is available.
Do `haxelib --global update haxelib` to get the latest version.
Downloading hxcpp-4,3,2.zip...
Download complete : 14313286 bytes in 3.6s (3806.7KB/s)
Installing hxcpp...
Current version is now 4.3.2
Done
A new version (4.1.0) of haxelib is available.
Do `haxelib --global update haxelib` to get the latest version.
Downloading lime-samples-7,0,0.zip...
Download complete : 8058725 bytes in 2.2s (3505.9KB/s)
Installing lime-samples...
Current version is now 7.0.0
Done
Do you want to install the "lime" command? [y/n/a] ? y
I could at least run lime test neko this way, but lime test linux gave Error: Source path "~/haxe/haxelib/hxcpp/4,3,2/bin/Linux64/std.dso" does not exist. Turns out hxcpp is too new — haxelib run lime setup installed the latest without asking.
Thanks to grep I found in my old project export/linux64/cpp/haxe/final.hxml:-D hxcpp=3.4.64.
So I ran haxelib set hxcpp 3.4.64, and lime test linux started to work! Kind of. There are lots of warnings, including deprecation notices from C++11. Then it ended with:
~/haxe/haxelib/hxcpp/3,4,64/src/hx/libs/std/Sys.cpp:34:19: fatal error: xlocale.h: No such file or directory
34 | #include <xlocale.h>
| ^ ~~~~~~~~~~
compilation terminated.
So: ln -s /usr/include/locale.h /usr/include/xlocale.h (thanks!)
Great, HelloWorld and DisplayABitmap work. (Well…they’re both just blank windows… but they compile and run.)
Now BunnyMark fails to compile, probably because lime-samples is way too new. Instead of 7.0.0, 3.4.0 might work better based on its timestamp.
Indeed, haxelib set lime-samples 3.4.0 and we’re in business. Still get the warnings, but HelloWorld is now a green window, and BunnyMark compiles and the bunnies flow.
But the next step, haxelib run openfl setup yields Error: Failed with error: No such Project : lime:2.9.1…sound familiar?
At least this works:
$ haxelib run openfl -version
.d88 88b. 888888b 888
d88P" "Y88b 888 888
888 888 888 888
888 888 88888b. .d88b. 88888b. 8888888 888
888 888 888 "88b d8P Y8b 888 "88b 888 888
888 888 888 888 88888888 888 888 888 888
Y88b. .d88P 888 d88P Y8b. 888 888 888 888
"Y88 88P" 88888P" "Y8888 888 888 888 "Y888P
888
888
OpenFL Command-Line Tools (3.6.1-LQBZc4)
Maybe I don’t need the setup—it’s just syntactic sugar anyway, letting you openfl ... instead of haxelib run openfl ....
I then tried haxelib set openfl-samples 4.0.0 so I could haxelib run openfl create PiratePig. But it needs actuate, probably 1.8.7 based on the time.
With that, it compiles, links, runs, and…segfaults. But with Neko it works OK. Hmm…
Maybe I’ll have luck with my game at this point. Let’s see. It’s back to crashdumper, random, systools, and my custom flixel.
Dang, my own library also defaulted to the latest flixel and openfl when running haxelib install flxscrollablearea. So, I just removed them again afterward.
Getting close
Now I can haxelib run openfl test linux and it nearly compiles. Still the C++11 deprecation warnings, and the first line is export/linux64/cpp/haxe/ApplicationMain.hx:89: characters 42-60 : Warning : Use programPath instead, but, overall, the vast majority compiles without complaint.
But then:
Link: ApplicationMain
Called from systools.Dialogs::boot systools/Dialogs.hx line 36
Called from systools.Loader::load systools/Loader.hx line 20
Called from cpp.Lib::load ~/projects/haxe-projects/haxe-manager/current/std/cpp/Lib.hx line 38
Error : Could not load module systools@systools_init__0
Turns out some kind soul has provided a newer build, so with a simple haxelib git systools git@github.com:jarnik/systools, it works!
Hooray!
So in the end, some minor tweaks and, most importantly, using Haxe Manager — thanks to checking slightly more widely for a solution — were what let me recompile my game on a more recent version of Linux, and it now works just fine.
I hope this story helps save someone time on their own Haxe projects.
