OpenTTD 15.0-beta3 30 Aug 4:00 PM (last month)
Did you have a lazy summer? Over at OpenTTD HQ, we did not!
We’ve continued to make improvements to OpenTTD, both very visible to player and under the hood. It’s time for another beta so you can help us test. Highlights include:
- Bridges can now be built over stations (NewGRF stations must be updated to support this)
- [NewGRF] Bridge GRFs can now have pillar information, so that a pillar can be skipped if it obstructs the tile below (rail, road, or station)
- Ships traveling in opposite directions try not to drive on the same line of tiles
- Purchase and picker menus can now be filtered using NewGRF Badges
- Settings and Options menus have been merged
- Scenario Editor has improvements for placing and expanding towns
- New buttons to expand town roads and buildings separately
- “Many random towns” button now prompts for the number of towns
- Picker window sprite previews can be increased in height, so you can see the entire skyscraper in the house picker (for example)
- The Infrastructure list window now has a scrollbar
- A variety of bugfixes
(As always, see the changelog for further details).
OpenTTD 15.0-beta2 12 Apr 4:00 PM (6 months ago)
April 1st has come and gone, but have no fear, for beta2 is here.
We’ve fixed a bunch more since beta1, and need more testing. Also, we added some new features! Highlights include:
- Snow-covered rocks are now visible
- Steep hills from map generation or heightmap import now get extra rocks
- Houses can be individually placed by players in the scenario editor and (if enabled) in-game, and can be protected from replacement during town growth
- Touchpad two-finger map scrolling for Windows
- NewGRF Badges
- A variety of bugfixes
(As always, see the changelog for further details).
What are badges, you ask? Badges are something add-on (NewGRF) authors can put on their content to help you find e.g. the vehicle or station you want to use. For example, various vehicle add-ons use a mix of different icons, overlay images or texts to indicate the power source or the suggested usage of a vehicle. With badges, this information can be represented in a clear way including built-in filtering, that is cohesive between different add-ons.
As this beta is the first release featuring badge support, there aren’t many updated add-ons yet, but we’re excited to see what authors do with the new feature.
Finally, the usual titlegame competition has been announced on the forums.
OpenTTD 15.0-beta1 23 Dec 2024 3:00 PM (10 months ago)
Ho, ho, ho, Santa is in town. And he’s brought a gift with him.
Here’s a shiny new package for you to unwrap, containing the first beta version of the upcoming OpenTTD 15 release.
It comes faster, with road waypoints, fewer passwords, better scenario editor, and hopefully with a lot less bugs. To be a bit less vague, here are some of the highlights:
- Improved picker windows for stations/waypoints/objects/and more. Oh, and that does include road waypoints.
- Better perfomance in various places, including faster path signals that are now green by default.
- Password-less network authentication using keys.
- Several improvements for NewGRFs regarding cargo support.
- Better scenario editor with a town data importer and manual house placement.
- A large tree fully hung with ornamental bugfixes.
- Sadly though, not everbody was good this year. The old NPF pathfinder was finally retired, after YAPF being the superior default for many, many years now.
There’s a lot of bugs left for you to find, though, guaranteed. Enjoy the beta, test all the things and report anything you find amiss, to make OpenTTD 15 the best OpenTTD 15 ever.
OpenTTD 14.1 2 May 2024 4:00 PM (last year)
Our players have played a lot of OpenTTD 14 since it was released a few weeks ago – in last week alone those who opted into our survey played 34,700 hours across 17,219 games.
In all those hours, you’ve found some bugs, and we’ve done our best to fix them. The first maintenance release for OpenTTD 14 fixes a multiplayer desync bug and cleans up a few rough edges with the new ship pathfinder and the unbunching feature. As always, there are plenty of other bugfixes, which you can find in the changelog.
OpenTTD 14.0 12 Apr 2024 4:00 PM (last year)
Welcome to 14.0!
OpenTTD’s first release was in March of 2004. Now twenty years later, we are proud to present to you: release 14.0.
And boy, what a release it is. Do I dare to say: this has been the biggest release yet?
Sometimes people try to tell us OpenTTD is dead. Well, 14.0 shows you it really isn’t. There are so many new goodies in 14.0, that I don’t even know where to start! Let’s go over a few, just to get you a feel for what to expect when you launch OpenTTD 14.0.
Read more about our birthday celebration here.
When one day isn’t a day
It is now possible to make the calendar time walk slower, up to a point it is frozen. This doesn’t influence vehicle movement at all, nor the amount of goods transported, town growth, etc. But it does influence vehicle introduction dates, inflation, and aging.
With this feature, we hope to enable different game-plays, where people want to have more time to build out their empire before vehicles expire, inflation gets too big, etc.
A brand new day! For some a bit slower than others.
Read more about this new feature in the devblog.
Ship Pathfinder
The ship pathfinder got some great love. It is now capable of directing ships over vast distances without the need for buoys. In fact, one could say, you don’t need buoys anymore at all. The ships will still find their way!
Finally, no more “Ship is lost” non-sense. This enables ship-only games.
Read more about this new feature in the devblog.
Unbunching
Do you also hate when busses start hugging each other on a route? And you did your best to ensure you send them out the depot with a good distance between them. But something must have happened down the road, and now they are very close to each other!
Introducing: unbunching.
A new feature that allows you to tell all vehicles in a shared order pool to keep their distance; automatically, done for you.
Read more about this new feature in the devblog.
Social Platform integration
OpenTTD 14.0 allows better integration with Social Platforms. This is totally optional, and has to be installed separately. But once done, OpenTTD can talk to platforms like Steam, Discord and GOG Galaxy.
For now, the functionality is limited: friends can see you play OpenTTD, if you are in a server or not, and what kind of map you are playing. For future versions, we intend to expand on that, making more possible. But, small steps.
Read more about this new feature in the devblog.
Survey support
To finally put a rest to endless debates about what setting is used and which is not, OpenTTD now has an opt-in survey tool built-in. Over time this will allow us, developers, to better understand how OpenTTD is played, so we can better anticipate what will be a hit or a miss.
You can, at any time, opt in or out from the survey.
Read more about this new feature in the devblog
GUI improvements / scalable font
There has been a huge effort in making all our windows just a tiny bit better. Fixing pixel-errors, moving things around a bit, fixing colouring issues.
But .. not less important, we now also have a built-in scalable font! Zephyris created a real (TTF) font heavily inspired by the sprite font. This means that scaling looks a lot better, especially for zoom levels like 1.5x.
We love it so much, it is the default when you launch the game! (but no worries, you can revert to the sprite font if you really want to).
Read more about this new feature in the devblog.
New players experience
First of all, there is now a Help and Manuals button in-game, that helps you through the basic information of the game. Bringing information closer to the user, and the game more accessible to everyone, has been a path we have been on for a while.
Second, we revised a lot of settings, what their default value should be. Many settings have had their defaults changed, mostly towards enabling modern features by default.
Lastly, we also renamed Cheats to Sandbox. We noticed people were struggling with using “cheats”, as we have been conditioned that cheats in games are bad. But Cheats in OpenTTD context is much more “play how you like” (read: Sandbox). So we renamed it, hopefully making it more clear to new players that it is perfectly okay to use these Sandbox options any time they like.
Hopefully this helps new players get into the game easier.
So much more …
In total OpenTTD 14.0 comes with roughly 40 new features, over 500 bug-fixes, 200 changes, and many many code improvements.
It took us 2,000 commits to get from 13.4 to 14.0, we touched 140,000 lines of code and removed 74,000 of them. This was all done by over 60 contributors.
Interested how this all came together? Check out our blog post explaining it all.
A great thank you to all who contributed, with code or support; it is truly appreciated! Here’s to another twenty years of OpenTTD development!
OpenTTD 14.0-RC3 23 Mar 2024 6:00 AM (last year)
Features: added. Gameplay: improved. Bugs: squashed. Final testing pre-release: probably, unless time is accidentally stopped.
Today we release 14.0-RC3, hopefully the last release candidate before the proper 14.0 release. Compared to the previous RC, some annoying bugs were fixed and the latest translation updates included.
The stoppable march of time 22 Mar 2024 4:00 PM (last year)
I was once the hottest new model on the street. Newspapers heralded my arrival in every town. The titans of industry were inspired to produce more goods when I visited their factories.
But as the years have passed, so do their eyes pass over me, to eye curiously my replacements. Will you try the new style, ma’am? It’s so much better than that old thing.
I’ve watched my friends grow old and die. My brother got caught in traffic and caused a horrific level crossing collision. I get sick more often, wheezing to a halt wherever I am. When I visit the doctor for a spot of renewal, they tell me, “Sorry, I can’t help. You’re too old.” When will I be autoreplaced?
Woe is the tale of the Balogh Goods Truck. But what if we could slow or pause the steady march of time? In OpenTTD 14, you can.
Time basics
Time in OpenTTD flows in three ways: The movement of vehicles, which we’ll call animation time. The keeping of economic records, including charging you running costs for vehicles, recording the graphs of your company’s income history, recording the production of industries and towns, and vehicle timetables. We’ll call this economy time. The passage of calendar time, affecting the introduction and expiry of vehicles, house styles, and the sub-arctic snowline (if using a variable snowline NewGRF). We’ll call this calendar time.
Let’s get the easiest out of the way first: we are leaving animation time alone.
Before OpenTTD 14, economy time was not tracked separately from calendar time. When players wanted to run steam engines for longer, they turned on “Vehicle never expire” or used the date cheat to rewind time when needed. Somebody wrote a Daylength patch, later included in JGR’s Patchpack, that slows down the combined calendar/economy time at the expense of many side effects.
But slowing the economy was not what old Balogh asked for.
The journey to real-time units
In late 2022, I began the work of splitting economy and calendar time. I was following a vision from my fellow developer nielsm, who proposed a novel solution to a problem that had vexed my previous attempts at this split.
The problem is this: If the economy does not follow the calendar, how do we display economy statistics like an industry’s “Production last month,” timetable arrival and departure dates, or the myriad other things that refer to time?
The ugliest solution is to display two dates. But OpenTTD’s user interface is confusing enough. Adding a second date is not a good user experience.
Nielsm’s proposal capitalizes on the fact that a day in OpenTTD is about two seconds, meaning that a 30-day month is about a minute. We can simply make this a bit more precise, then display economy time in real seconds and minutes!
The first step was making the seconds-per-day a bit more precise. The smallest unit of time in OpenTTD is a tick, of which there are 74 in a standard day. Prior to OpenTTD 14, one tick lasted 30 milliseconds, making a day 2.22 seconds. That’s close, but not quite close enough when multiplied out to a month or a year. But in TTD, one tick is 27 milliseconds, making a day 1.998 seconds. That’s about as perfect as we can get. In OpenTTD, we’ve gone back to this original TTD value to better align to real-time seconds.
With this the “amount of goods produced in an in-game month” is about the same as the “amount of goods produced in one realtime minute”. A “vehicle service interval of 100 in-game days” is about the same as a “vehicle service interval of 200 real-time seconds”.
Of course, not everyone will want to play OpenTTD using real-time units, so the new feature is an opt-in setting called Timekeeping units.
Calendar units are the classic OpenTTD experience, with no changes. Wallclock units use minutes anywhere months are normally displayed, with 12-minute periods substituting for years.
Where a date would normally be displayed (timetable start date, subsidy expiration date, etc.), the game shows an offset in seconds or minutes from the current moment. Timetables are expressed in seconds.
It’s also possible to select seconds as the timetable units in any timekeeping mode, in the Settings menu.
How to slow down time
There are two new settings used to control the flow of calendar time.
-
Timekeeping: Select the timekeeping units of the game. Calendar is the traditional OpenTTD experience and you won’t notice any changes. Wallclock displays the new real-time units and is required to adjust the rate of calendar progression.
-
Minutes per year: Select how long a year lasts. The default is 12 minutes. Set this to 0 to freeze calendar time completely. The maximum is 10,080 minutes, which is one week of real time!
The timekeeping mode can only be changed before starting a game, so choose wisely! Minutes per year can be changed at any time.
What about cargo scaling?
The economy time always progresses at the usual rate, so it has none of the side effects of the Daylength patch. That said, one of the side effects that players like is the reduced cargo production of houses and industries. We’ve added separate settings to scale these, and you can do this in any timekeeping mode.
You may notice some differences with NewGRF industries. Industries normally produce cargo every 256 ticks, and for base game and simple NewGRF industries we simply scale the amount of cargo produced. For NewGRF industries which calculate their own cargo consumption and production, this could produce unwanted results. For these, we instead scale the frequency of this production action, so at a lower scale the industry performs this action less often.
Some industry NewGRFs will have inaccurate helptext because of this change, as they will with wallclock timekeeping — for example, some industries increase production when supplied with boost cargos every three months. NewGRF authors will have to update their GRFs to display properly in the new modes, although of course there is no change required in Calendar timekeeping mode with 100% cargo scaling.
Cleaving time in two
Let’s get back to new time features. To actually implement these, we had to work backwards from the ending point. In order to stay compatible with calendar timekeeping units, the economy time units need to take the same form of years, months, and days, so that in calendar mode they follow the calendar date exactly. In wallclock mode, however, they diverge: an economy-month always has 30 days to always last one minute, so an economy-year always lasts 12 minutes. In contrast: the calendar-year lasts 365 or 366 days, so 12 minutes and 10 or 12 seconds.
This difference between calendar and economy dates is why players are not allowed to change timekeeping units in the middle of a game. OpenTTD’s 20-year-old codebase contains a lot of assumptions about how time works and does not react nicely to synchronizing or un-synchronizing two date systems. For example, if we are in calendar mode on the 31st day of the month and switch to wallclock mode, that day would not exist. The date would have to be shifted, but that can make company finance data stop being recorded for two thousand years, crash the game when the Cargo Distribution linkgraph checks if it’s time to update, or (worst of all) cause some other bug I haven’t discovered! Maybe someday, another contributor will write the code to nicely convert between the two modes and enable the timekeeping mode to be changed mid-game, but now is not that time nor am I that contributor.
Tech debt comes knocking
Of course, there was a lot of technical debt to be paid off before we could even get here. The calendar had to be cleaved in two and every reference to it evaluated to determine if it should continue to reference the calendar or be changed to the new economy timer.
My fellow developer TrueBrain was instrumental in helping me with this step, writing a new strongly-typed timer system to help catch any errors. Strongly-typed does not refer to how hard you bang the keyboard, but to a type of variable that refuses to accept the wrong type of data. So if I try to pass a calendar date to a variable that stores an economy date, the compiler will give me an error and refuse to compile until I fix it.
With a codebase as old as OpenTTD, anytime we touch something it provides an opportunity to leave the code better than we found it. With such a wide-ranging project as changing how time works, I touched most of the codebase and found many opportunities to refactor, fix, or clean up old code.
Conclusion
I have been working on this project for over a year, often putting in 20 hours a week. Like with most development, my motivation is a mix of wanting to play with this feature myself, and simply enjoying the challenge. Why optimize my rail network when I can optimize the code that makes it possible?
This is the last Dev Diary before we release OpenTTD 14.0. Thanks for joining us to learn more about the OpenTTD development process. If you’d like to learn more, we hope you’ll join us in the official OpenTTD Discord or on GitHub. Stay tuned for a release post around April 1st!
OpenTTD 14.0-RC2 16 Mar 2024 5:00 AM (last year)
Today we release 14.0-RC2, the second release candidate for the upcoming release. We’ve fixed a whole bunch of issues found in the first release candidate. However, there is a good chance there are still some issues out there that haven’t been found yet. Please help us hunt those down, and if you find one, you can report them here.
Translation updates are coming in every day, most languages are complete or are nearing completion. If you are a translator, please see if your language has any missing translations. We greatly appreciate your contribution!
Survey & HTTPS & Infra 15 Mar 2024 4:00 PM (last year)
With OpenTTD 14, we introduce an opt-in survey system; a method where we can finally ground our debates with facts! Additionally, BaNaNaS content will use HTTPS (instead of HTTP), and many more small infra-related changes.
Time to chat about this for a bit.
Survey
For years now, we can have very passionate discussions about one gameplay style or the other. And often, sooner or later, someone will drop the “that is how 90% of players play” argument in there.
Do they have anything to back up that 90%? Of course not. It is an emotional argument in an often emotional discussion (as we all greatly care about the game). But it doesn’t actually help the discussion further, as the other party goes: “no! That is how only 10% plays!”. And as such, the discussion dies off and we are unable to move forward.
To finally settle which 90% was correct, OpenTTD 14.0 introduces an opt-in survey system where, after every game, a report is sent about the settings used in that game. This is enabled in the 14.0 betas and is already giving very solid insights how the game is actually played; instead of how we all think it is. It is extremely valuable for further development, to better balance where to put focus.
It is extremely important to us that this survey mechanism is both transparent and privacy friendly. As such, a lot of effort has gone into ensuring it is both.
When you first start OpenTTD 14, you are greeted by a dialog asking you if you want to opt-in to the survey. If you don’t (either close the dialog or decline), no survey information will ever be sent to us. But if you do, at the end of every game, a small file is transmitted to us.
What is in it you ask? For that we created a “Preview” button, which shows exactly what information would be sent. Nothing more, nothing less. What is in there, is what we receive.
And if you change your mind (or want to see the Preview again), under “Game Options” you can change your choice at any time. No strings attached; no nag-screens “are you sure?!”. We highly appreciate it if you enable it. But if you don’t, that is perfectly fine too.
Once received, the survey is processed and checked whether it is a valid survey, for what version, and stored in an efficient way in the backend. Every once in a while an analyzer runs over all the surveys to produce survey results. They can be seen here with all their details made available.
There are all kinds of protection against over-representing one population or the other, although we still have work to do to tune that system better. But, as more surveys arrive, we also have more information to improve on this.
So if you are curious how your playstyle is versus many other people: check out the summaries and get an impression!
Just be warned, and I have to remind people about this almost daily: the summaries are a snapshot of that moment in time and can over-represent one group or the other. Until the actual 14.0 is released, be aware that the sample size is rather small and summaries can differ widely from week to week. In other words: be careful to not make wrong conclusions on something, based on a result of a single week.
Popularity
Often it is rumoured that OpenTTD is dead. We don’t see any evidence of that, however you slice it.
OpenTTD development is doing very well, as the new feature list for 14.0 is showing. But also: the player-base has not declined.
Take for example the amount of concurrent players on Steam. In the last 3 years this amount has grown, from ~600 to ~1000. That is a huge increase.
From internal Steam financial information (yes, free games also have financial information), we can see that the game has been installed by almost 2 million unique users in the last 3 years. That is so many more players than I ever expected would play this game.
More information from Steam: we can also see how popular our news posts are. Our 13.0 release post, for example, has been read by 150,000 unique players (with over 2.5M impressions). That is a lot; especially for a free game that is 20 years old!
When we look at our wiki, we see that weekly ~100,000 unique players visit the wiki for one page or the other. Most popular is, of course, the tutorial section on the wiki.
And although normally we get around 100 unique visitors per hour on our main website, sometimes there are posts like this on popular websites. These posts increase that number to over 12,000 unique visitors in just an hour.
All in all, OpenTTD has been as popular as it has ever been and many players still play the game day to day. In fact, we see a (slow) increase in popularity, especially when looking at bandwidth usage. More on this in a bit.
BaNaNaS via HTTPS
When BaNaNaS was added back in OpenTTD 0.7, it was done via a custom TCP protocol. Very quickly we found out that it was bandwidth hungry and we needed a good way to distribute this load.
HTTP was added in OpenTTD 1.0 for this goal and most downloads you do today are actually done via HTTP. But, as HTTP is less and less common these days, and more and more companies start to block communication over HTTP, it was time to implement HTTPS.
Where our HTTP downloader was a custom build, with all problems that come with it, for HTTPS we use either WinHttp (Windows) or libcurl (Linux / MacOS). This means that someone else took care of all the complexity of downloads via HTTPS and we have a simple interface to download a file.
With this change we hope many more people can download BaNaNaS content via HTTPS (instead of TCP); this is not only a lot quicker (as HTTPS uses Cloudflare, which ensures downloads come from a local Point of Presence (POP)), it is also a lot cheaper for us if you do.
BaNaNaS bandwidth bill
Hooking in to the above, the move to HTTPS also has another goal: about 10% of our bandwidth for BaNaNaS is till this day done via TCP. Although the HTTP(S) downloads are free of charge (thanks to Cloudflare), the TCP downloads are not.
To be more exact, as they are served by AWS, it costs 0.90 dollar per GB transferred. And this grows to a large number very quickly; over 50% of our monthly bill is only for BaNaNaS downloads traffic.
Now there are many ways to mitigate this, ranging from “migrate away from AWS” to “just disable TCP downloads”. But for now, I have good hope that the change to HTTPS will move even more traffic away from TCP. Related to that, we also removed the setting that forced usage of TCP over HTTPS: OpenTTD now always tries HTTPS first; and only if that fails, it falls back to TCP. Hopefully that sufficiently reduces the bandwidth bill.
If not, we will need to take action on this. I keep hoping Cloudflare releases the ability to route TCP (and not just HTTP(S)) via their infrastructure; they keep teasing this in their blog-posts, but they have since 2021. So at a certain point I have to give up that hope and find a better solution myself.
Infrastructure
Our migration in 2023 was a great success and continues to be so. Deployments have never been easier and maintenance has been a breeze. Auto-healing and incident resolving work as expected and in general, Nomad has been a good fit. This combined with our new GitHub workflows, there is a clear noticeable difference in how much time is required to keep everything running (for the better, ofc).
Our Cloudflare bill has been very stable; which is very good for a project like OpenTTD, as predictability means we can try new things, and see their impact.
For example, this year we added a Symbols server. This server collects all the debug-files for all our OSes, making it easier/faster for us to delve into crash dumps people send us. Before the Symbols server there were a lot of manual actions required to open up a crash dump; now it is just pressing a single button and waiting for a bit.
We also added much more caching to the wiki. As wiki pages do not change all that often, we can cache them pretty aggressively. This means that if you go to our wiki now, Cloudflare serves almost every page from cache. In the background it does validate if any content is out-of-date. If any out-of-date content is found, the cache is updated with the latest. For you this means a much faster website; for us it means a lower AWS bandwidth bill. Around 5% lower, in case you are curious.
Lastly, a bunch of backend services gained a Prometheus metrics endpoint, meaning they are easier to monitor, what they are up to, and how they are used. This for example allows us to see how many (percentage-wise) OpenTTD players have an IPv6 enabled connection (answer: 23%) and what versions of OpenTTD are connecting to BaNaNaS. This helps us understand where to put effort for the backend services and what is “working as expected”.
Donations
As you might know, the OpenTTD infrastructure is fully paid by donations. And for years now, we didn’t need to do a fundraiser, as you all graciously donate enough money in a year to pay the bills. What helped, especially in the earlier years, that we were often sponsored by companies with infrastructure related sponsorships.
These days we enjoy fewer sponsorships. This is not always a bad thing; if you are a paying customer, companies are more likely to help you out. Especially with infrastructure this hasn’t always been the case during sponsorships.
That said, two companies do deserve a shout-out, as they do sponsor us still and we greatly appreciate them.
Pulimi, to help manage our infrastructure. 1Password, to ensure we can store all OpenTTD related passwords safely.
We told them by email, but also here: thank you so much for your on-going sponsorship; it is truly appreciated.
As for the rest of our infrastructure, we are on our own and the reserve we had is slowly running dry. This means it is not unlikely we will once again need a fundraiser sooner or later. When that moment arrives, we will let you know. Till then, I would like to thank you all for your donations; we really do appreciate them and they keep the (virtual) lights on.
Future ahead
As for the future, there is a lot of interest for Cloud Saves: something we are still looking into. With Cloud Saves you would be able to save your savegame in the cloud and continue playing it from where-ever you are. And as with all services OpenTTD offers, it will not be only available for Steam players, but for everyone (regardless of what platform you use).
The main trouble here is to figure out what the cost per player will be and making sure we can actually afford it year-to-year. As you can read above, although we are financially healthy, we have to ensure we stay that way. Which means it will mostly depend on whether the changes in 14.0 help in reducing the BaNaNaS bandwidth bill, before we start extending our services to include new (not-cost-free) services.
The biggest concern we current have: how big will the average savegame be? Do most players play small maps? Or very large maps? This is not an easy answer, but will be answered by the survey results.
To give you a rough estimate: the cost for Cloud Saves will be roughly 0.02 dollar per GB per month stored. If we assume an average savegame to be ~10MB and we allow storing 10 saves per player, that would mean the average player will cost us 0.002 dollar per month. Or 0.02 dollar per year per player. On its own not a lot, but if you remember from earlier: we have ~1000 active players per day … it could add up to a lot of money really fast. So we need better metrics to know how many players to expect. And whether ~10MB is fair.
As you can read, lots to do here before we can even start to think about the technical implementation. These things just don’t come for free!
More about OpenTTD 14
This post is part of the series of dev diaries about big new features coming in OpenTTD 14. Next week, we will experience some time dilation, and dive into one of the biggest new features in 14: timekeeping!
How the sausage is made, or: How OpenTTD is developed 8 Mar 2024 3:00 PM (last year)
Ever wondered how a new OpenTTD release is made? How we decide what features to include and what to reject or how some people seem to know the “future” before you? Curious what it means that OpenTTD is Open Source? Or maybe you’ve even wondered what it takes to get your idea included in OpenTTD?
In this post, you’ll get a peek behind the quite transparent curtain and learn more on how OpenTTD is developed.
What’s source anyway?
Let’s start with the meaning of Open Source, as the source (or source code) of a computer program really is the source of it all. In a nutshell, the source code of a computer program is a form that is a lot easier for humans to understand than the instructions the computer is executing. For OpenTTD, we use the programming language C++. This source code is then transformed by a piece of software called a compiler into a binary form that the processor in your device can run. The binary is then combined with various other data files to create the game you can download from our website or for example get via Steam.
While every computer program has a source, not every program is Open Source.
Open Source requires that the source code is freely available and allows modifications by other parties.
You can look at the source code of OpenTTD right now in our git repository on GitHub (but please come back here afterwards).
To be able to track changes to the source code, we use a version control system called git.
It shows us each and every individual change (called commit in git) that was made to the game since its inception 20 years ago.
Being able to see old changes is quite useful when trying to fix problems or figuring out why something was done the way it was done.
The list of commits is also what you need to watch to gain “magical” knowledge about things that are coming in future releases of OpenTTD.
Our GitHub page is also where things are tracked that are not yet part of the game and still in development. You might have encountered it before as the place where you can report an issue with the game. Besides the git repository of the game itself, GitHub also hosts various other git repositories related to the game, like for example OpenGFX, NewGRF development tools or various back-end infrastructure.
How a feature is born
Now that we know where development happens, let’s look at how it happens.
Each and every change to OpenTTD, no matter if it is a large new feature or just a simple one-line bug fix, starts with someone choosing to spend their time on it. As the source code of the game is freely accessible, this can be literally anyone, for a multitude of reasons.
Many features come about because someone has an idea how their game could be better and decides to work on it. Bugs are usually fixed because someone either feels responsible for it in some way or is just plain annoyed by it. But really, everybody has a different personal motivation. There’s one reason you’re not going to find though: Because the boss said so, or because it is on the big company road map. OpenTTD is purely developed by volunteers, there’s no company, foundation, or other controlling central entity behind it.
Ideas are often discussed in the development channel of the official OpenTTD Discord. It’s a good way to gauge interest, exchange ideas and chat about issues before spending any significant time on something. It’s also a fast way to get help or inspiration when stuck on something while developing for OpenTTD. While activity in the channel does vary over the day, someone is usually always awake.
Now let’s assume that the person is in fact developing a feature (like for example the new ship pathfinder) and has completed the initial development. Since OpenTTD moved to its current home on GitHub almost 6 years ago, we’ve adopted a fixed process for any change to OpenTTD. The first step in the process is to open what is called a pull request (PR) on GitHub with the proposed changes to the source code in the form of git commits. Additionally, we require a description of the changes following a specific template to make sure nothing important is left out.
Whenever a PR is opened, various automatic checks run to verify some formal things, but while those checks are running, other contributors can already begin to review and comment on the pull request. The automated checks include what is called continuous integration (CI), which ensures the PR actually compiles on all platforms we support and not just the one the author is working on.
Automated checks are well and good, but so far they can’t replace a human code review. In code review, other people look at the code and try to find any issues or spots that could be improved. This can be everything from validating functionality, checking UI and user experience, to proof-reading code comments. For a large change like the ship pathfinder, we even organized a multi-player test game to check for any dreaded desync problems.
Usually, this results in a back and forth between the author updating the PR and the reviewers looking again until the PR is either declared good or rejected. Unfortunately, not every author has the motivation to follow this until the end, so it happens that PRs just kind of fizzle off without reaching a finished state. Up to here, everybody can pitch in with reviewing a PR, but actually approving or rejecting a PR requires a decision by someone that has commit access to the OpenTTD repository.
PR aren’t actually outright rejected that often. Reasons for rejection might be that the PR is contrary to the goals of the OpenTTD project, like trying to add content that could be very well done as an add-on NewGRF. Sometimes it also happens that a PR is meant well, but just not the right solution for a particular issue because it might affect some other part of the game negatively.
For approving a PR we’re quite strict, maybe stricter than many other projects out there. The OpenTTD project has a rigid code formatting style guide and commit message format. We’re also unlikely to accept a PR with known problems or unfinished parts. For example, the game is translated to many languages, including some that are written right-to-left, and any UI code needs to be coded to work for all languages and not just English. Finally, GitHub will prevent approving a PR if any of the CI checks have failed.
While all this might sometimes appear to be overly pedantic and petty, a strict quality control is important to make sure the game can continue for the next 20 years. Everybody has probably noticed that everything, from your desk to the storage shelf in the shed, seems to always descend into chaos unless cleaned up regularly, and code is no exception to this. If we wouldn’t care about code quality, the source code would probably be a hot mess after 20 years and full of bugs, making any new features really difficult.
One thing of note here is that no author can self-approve a PR, even if they have commit access to OpenTTD. In this sense, all PR authors are treated equal and need to follow the same process to get a PR approved.
Finally, if a PR has the coveted approval, it can be merged into the main code base. This can be done by the person approving it, but especially for bigger changes we like the merge to be done by a second set of eyes. From that point in time it is part of the game, but not yet in the version you are playing right now. For this it still needs a release, which we’ll cover in the next section.
The release process
There are two kinds of releases we make for OpenTTD. The first one is the “nightly”. This is automatically made out of a snapshot of the source code each day. The nightly release uses almost the same CI workflow as described for the PR checks, except that the result is published on our website and on Steam.
The other kind of releases are the interesting ones, the ones that get a proper version assigned, like the upcoming OpenTTD 14.0. These happen whenever we think there are enough new things, improvements, and bug fixes since we made the last major release.
If you’ve been following the news on OpenTTD, you’ll have noticed that we’ve already released some versions with a 14 in it, but yet we’re telling you that 14.0 will come soon. You might ask: What’s up with that? Unfortunately, programmers are just as human as you are, and thus it is very likely that OpenTTD contains numerous bugs at any given time.
To make sure we can give you the best possible game to play, all major releases follow a specific path, which is shown in the image on the right. It starts with one or more so-called beta releases. A beta release is taken directly from the main source code at a certain point in time. The main purpose of beta releases is to gather feedback from players about new features and to find any issues with the game. Beta releases are still marked as testing releases and are usually not included in Linux distributions or the normal Steam updates, thus depend on players willingly trying a potentially unstable game version.
Even if it’s directly taken from the main source, there’s still a lot of stuff that needs to be done even for a beta release. Someone has to look over all the changes since the last release and write the changelog. The website news post has to be written, posts for social media like Discord or Reddit prepared, and an image for the Steam news post drawn. When everything is ready, a so-called tag is created that marks a specific commit. With this tag, some more automated CI workflows compile binaries for the various platforms we support and upload them to our website and the other distribution platforms. When the CI is done, the news post can be published and the social media posts made so you will actually know that there is a new release.
As it is rare to not find any issues, there are usually multiple beta releases until the number of issues encountered by players drops sufficiently. The next major step in the release process after the betas is branching and feature freeze. Branching means that the source code for the release is split off from the main source code and will no longer automatically track changes made there. Bug fixes are still applied to the branched code, but new features are not. This helps to prevent last-minute problems in the release versions.
After branching, the testing releases continue with one or more Release Candidates (RCs). RCs are similar to the beta version in that issues are still expected, but as no new features are applied anymore, the issue count should go down, not up. The actual release is basically done the same way as a beta: prepare a changelog, all the social news, make a tag, wait for the CI, and finally tell the world.
When no more major issues are found in the latest RC, the time for the first proper release has come. There’s really not much difference in the release process from the RCs, except that this version is not marked as a testing release and thus will be for example picked up by the automatic update on Steam or GOG. We do spend more time on the changelog, news post and social media posts for the release, as a proper non-testing release has a much larger audience than the preceding testing releases.
Inevitably, a larger audience is also better in finding bugs, which is why the maintenance releases exists. Maintenance releases are the X.1, X.2 and so on versions. They do not include new features over the major X.0 release, but fix whatever issues are found.
Thanks to a lot of behind the scenes magic done by some wizards on the CI workflows, the most time-consuming part of any release is actually creating and publishing all the public-facing text and information. Compiling the binary and uploading the result to the various distribution platforms requires surprisingly few mouse clicks to trigger. This wasn’t always the case though. When OpenTTD was still young and sites like GitHub were just not a thing, the various binaries for e.g. Windows, Linux or macOS were hand-compiled by different people, manually collected and uploaded. Often enough that meant that it literally took several days until a release could be downloaded for all platforms. Compared to that, the current automatic CI workflows really do feel like awesome magic.
Getting personal
If you’ve been diligently reading till now, you might have noticed that I haven’t used the word “developer” so far. Instead, I’ve mostly talked about a generic someone. So what’s up with that?
While many people think that the individuals with merge permissions are “the” developers of OpenTTD, this isn’t really true at all. GitHub provides various statistics for each project, and lists 169 contributors to OpenTTD at the time of writing this. And this number is still much too low, as it is for example missing many contributors from before we were on GitHub, and doesn’t count our language translators since their changes are committed by a bot. These hundreds of individuals are the real developers of OpenTTD and the people with commit access are more akin to housekeeping or maintenance.
When you read something like “Why haven’t the devs included X yet?”, it is very rarely because someone with merge permissions said “nope”, even if the questions is often phrased to imply this. It is not included yet because nobody volunteered their time for it.
And that really is the gist of it. OpenTTD is not backed by a big company or some other institution, but purely by volunteer work. OpenTTD depends on donations. This does include monetary donations needed to pay for infrastructure and services, like our website and all the automated systems we’ve described here. But probably even more important, it also includes time donations. And time is what is missing most of time (ahem). So the best way to get X included is to donate some of your time.
Final thoughts
OpenTTD would not have made to 20 years if it weren’t for the hundreds of people that chose to donate some of their time to the project. As such, we are very thankful for everybody who contributed something, no matter how small or big. Without people who contribute code, triage issues on GitHub, make translations, create content like NewGRFs or AIs, or help answer questions on places like the official Discord, Reddit, or Steam forums, OpenTTD would not be where it is right now. And without you playing the game, it wouldn’t be here either.
More about OpenTTD 14
This post is part of the series of dev diaries about big new features coming in OpenTTD 14. Next week, we’ll get a survey of some more behind-the-scenes work that helps us to better understand what players like about OpenTTD.