FITS Guid

What is FITS Guid?

Throughout the last 100 or so years, professional and amateur astronomers have taken several billion images of the night skies. Most of these images are currently available in a digital format, and over 90% of those digital images are in the FITS format.

"Flexible Image Transport System (FITS) is an open standard defining a digital file format useful for storage, transmission and processing of data formatted as multi-dimensional arrays (for example a 2D image), or tables. FITS is the most commonly used digital file format in astronomy" - from Wikipedia.

Although FITS is the de-facto standard used by professional astronomers and most serious amateurs, it is old, messy, and not well defined, and it's a standard which is difficult for developers to understand and implement. The goal of this site therefore is to help our fellow developers to overcome many of these difficulties by explaining the standard in detail, by suggesting implementation strategies, and by providing an exhaustive list of useful resources. And we won't shy away from explaining general astronomical image processing algorithms and introducing the readers of this site to other related astronomical standards.

FITS is HUGE!

For most muggles (and most developers are muggles, including the authors of this site), astronomical images are just lossless science-grade 2D images of the sky. If only the Universe was that simple!

One can describe FITS firstly in its capability to combine multiple images into a single file, each representing data for up to 999 dimensions (and you thought Einstein was complicating the world by introducing time as the forth dimension). In addition, FITS files are capable of encapsulating binary and ascii tables of many kinds of data - spectral and photometric data, strange satellite data (like data from X-Ray satellites), data from massive radio-astronomy arrays, and entire catalogues of observation data. The data can be compressed (but, of course, in an astronomical compression method - gzip is unknown here). Oh, and astronomers talk about something called WCS (World Coordinate System) and time coordinates! Of course, for astronomers "World" is just the Galaxy in a barycentric coordinate system. Poor old Euclid from our school textbooks! And we, poor but not that old developers, are supposed to make sense of all this astro-mumbo-jumbo and even code it in super fast, well tested, and bullet proof programs; these programs should never ever crash-land the next Mars mission into Jupiter!

Finally, FITS is a 1970s standard, developed during the 1980s, and it is a format that is supposed to work on the Starship Voyager's computer.

So, to summarise, this is a site for astro dev muggles written, by astro dev muggles that happen to have a bit of experience.

FITS Guide is more than just FITS explained

FITS is an archaic but useful file format that needs to be dissected in order to be understood by software developers. But the most important question: what do we do with all this astronomical data? First, we the developers need to prepare the data in a format that astronomers can analyse easily and intuitively. Then, there are many routine procedures that could be and should be automated. A good example of such a procedure is the image pipeline, in which a large set of various calibration data and multiple noisy images produce a final image which is capable of displaying fine details of faint deep-sky objects.

In FITS Guide, we will cover the software aspects of image processing (and there is a lot to be covered). We will discuss how to identify individual objects from an image, how to interpret satellite and radio astronomy data, how to calculate orbital elements of near Earth objects, and much more.

In addition, FITS files often contain data about the instruments involved in the data gathering and the surrounding environment. Unfortunately the FITS standard imposes very few requirements on how to describe instruments and their environment, and it is often very difficult to parse this important information automatically. We will discuss how to build universal model objects and how to parse the data in a more reliable manner.

And finally - errors! Many FITS files are broken or do not strictly comply to the standard. We will describe a few tricks on how to cope with errors and rewrite the files in a standard-complying way.

About the site

We are dividing the site in several major sections:

Who are the people behind FITS Guide?

This site is developed and maintained by the StarCluster team. After being involved in the development of robotic telescope software projects and related communication standards used by observatories to talk to each other, our team decided in 2019 to develop a new, modern package of astronomy applications. A substantial part of our work is related to astronomical image processing, so of course we work with many different variants of FITS files.

Soon after we started, we recognised that the frameworks and software libraries implementing the FITS standard are either incomplete or just slow and with horribly outdated architecture - or both. These frameworks and software libraries are definitely not suitable for our modern mobile devices. Therefore, we made the decision to start from scratch. We are currently developing several Swift based frameworks for reading, modifying, and creating new FITS files from raw data or to store results of image processing and astronomical surveys using FITS file format. This site captures our experience developing these frameworks and apps.

Why not open source?

So, by now you are probably thinking: why make all this effort to write about how to develop software and not make it open source? Well, first of all, most of the software being forged is reusable for us but not for other software systems, because not only is it about FITS, but it's also about how FITS is stored in specialised, proprietary file archives, how it is created by CCD cameras, how it is modified by the observatory scheduler and enriched with additional environment data with instruments like a weather station. Secondly, we made the decision to write our software for Apple devices - Mac, iPhone, iPad, Apple Watch. So the language of choice is Swift, and this makes it not very useful for non-Apple computers or mobile devices. Lastly, our team is working with other teams, among which governmental organisations could potentially be a part of in the future. Therefore, it is possible that future contractual obligations will prevent us from open-sourcing our frameworks.

So we decided that the best way to help our fellow developers is to describe how to build a FITS framework.

We will be extremely thankful for any feedback, and we will try to answer any questions you may have.

The process of creating and maintaining the site

This portal is created with the static site generator called Publish by John Sundell. We enhanced Publish in many ways to accommodate the needs of the site, and we hope to be able to contribute to it later, as well.

When we hopefully meet John in person, he will get a few well-deserved 🍷🥃🍺! Thank you, John!