Using the railway network as a flatbed scanner

Aug 18, 2026 07:43 PM - 1 day ago 4

Using the railway web arsenic a flatbed scanner

August 17th, 2026 — 4,600 words

Over the past fewer months, I've been moving connected utilizing an business linear scanning camera to return very wide photos retired of trains and ferries. Getting it moving has been rather the challenge, but I deliberation the results speak for themselves.

taken connected the San Francisco to Oakland ferry successful February 2026 (56,894x2,048 pixel grayscale image); scroll to zoom successful and click and resistance to move

More pictures are connected show successful the gallery.

I presented a talk connected this task astatine EMFcamp 2026, which you tin watch beneath aliases publication connected for the aforesaid communicative successful much detail:

The process of capturing an image for illustration the 1 of the instrumentality larboard above.

The camera is pointed retired of a moving conveyance and is perpetually capturing a azygous vertical statement kinda for illustration these grayscale ones successful the diagram, but a batch thinner. As the camera moves, what precisely it sees is changing. If I seizure the lines from the camera quickly capable and stitch them together, I tin nutrient a complete-looking image. It's a bit much analyzable than that and getting the results looking bully was alternatively tricky, but that's the main thought down it.

Back successful the 1990s, integer camera sensor exertion hadn't caught up to the size and effective solution of mean and ample format film, truthful digital scanning backs were developed. They seizure a high-resolution image without needing a elephantine grid of pixels by moving a azygous statement of pixels (or 3 lines for color) crossed the frame. In the intervening years, image sensors person gotten beautiful large (there's moreover one that covers 4x5" ample format nowadays), but this attack is still cheaper to build for ample formats than a elephantine sensor.

I'd been reasoning astir building my ain integer scanning backmost for my ample format camera for a while, but I've ne'er rather gotten astir to it because building thing to equine decently connected my camera seemed excessively daunting. (Buying 1 could person been an option, but ones from the 1990s still spell for thousands of dollars connected ebay and require reconstructing a computing situation of a akin vintage to use.) Late past year, I was watching a video connected Gigawipf's mean format scanning camera build and abruptly thought: "what if the full camera moved and the taxable didn't?" and decided to springiness it a shot.

Loading movie into my ample format camera connected apical of a upland successful Vermont because I'm allergic to doing photography successful a normal way. (The resulting pictures from that travel are here.)

I recovered immoderate erstwhile photos successful the aforesaid vein (the Scannoramic project, John Hikerbiker's experiment, Daniel Lawrence Lu's reversal of his stationary camera, and Martin Liebscher's very absorbing movie shots), but the results seemed for illustration they could beryllium improved upon. Surely taking the velocity of mobility into relationship and getting cleaner results wouldn't beryllium excessively hard, right?

On the nighttime I thought up this "big scanner" concept, I had to springiness it a shot. It was a spot precocious to spell retired and drawback a train, truthful I scanned my sofa instead.

I group my telephone connected my agency chair and slow pushed it on arsenic it captured a video. I past wrote immoderate really slapdash codification (which I americium choosing not to stock present to protect my readers) to drawback the leftmost file (a "slit") of each framework and harvester them into an image.

My comments included lyrics from "Future Me Hates Me" by The Beths, which became thing of a self-fulfilling prophecy erstwhile I started penning a postprocessor for the adjacent type of the camera loosely based connected that codification and cursed my decisions.

It looks vaguely for illustration my sofa, but it's alternatively squished and the creation connected the wall is unintelligible. Surely I tin do better.

I messed astir pinch the postprocessing and doubled each column, which makes it look little squished, but it's still a messiness because I wasn't pushing the chair astatine a peculiarly accordant speed.

I knew from the commencement that I'd request to measurement the velocity somehow, but I was naïvely hoping that I wouldn't request to measurement it that good and could simply fudge it. This image, however, shows that moreover mini variations of velocity matter. This was my first glimpse into really overmuch of a symptom dealing pinch velocity would move retired to be.

For my adjacent trick, I took a thrust connected the MBTA orangish line. I taped my aged telephone to the spot to usage its accelerometer and held my existent telephone to the window, making judge to move the framework complaint up each the measurement to 60 fps.

The accelerometer information wasn't very useful and was moreover little truthful erstwhile I took an integral to get velocity.
If I retrieve correctly, y was the axis of the train's movement, but the information is truthful noisy that the train was apparently moving backwards astatine the end.

The consequence looks interesting, though, but I decidedly request much lines if I want a decently intelligible image.

While I was getting fresh for EMFcamp, I noticed another talk connected the schedule by Tim Jacobs (better known online arsenic mitxela) that was besides astir slit scan cameras and started to interest we'd some done the aforesaid thing. (He ran up to maine aft my talk to show maine he'd also worried this.) His talk started successful the aforesaid way, pinch taking a slit from a video, but he ended up making really cool and trippy animations by going done each imaginable slit position for a fixed video.

My root for much lines per 2nd ended up being the Basler ruL2048-19gm, designed to beryllium pointed astatine fast-moving conveyor belts. The oddly-capitalized sanction comes from its expertise to publication retired its 1x2048 pixel image sensor conscionable awkward of 19,000 times per second.

These capabilities travel astatine a price, however; marque new, the manufacturer's lowest-spec existent models spell for astir US$700. Thankfully for my wallet, I recovered excavation connected ebay for a tenth of that.

The value is besides measured successful light. since it's capturing truthful quickly (the slowest vulnerability clip is 1/100s), it needs a lot of light. I tin only sprout successful the daytime, and each but the brightest stations and tunnels are disconnected limits to me.

To my surprise, having dealt pinch vendorware before, Basler conscionable fto maine download the SDK without a support statement aliases impervious of purchase. The astir caller type besides still supports this camera from 2013, which is little astonishing but is still convenient.

The camera communicates pinch the machine complete a gigabit ethernet nexus and the package finds it automatically arsenic agelong arsenic the applicable interface is group up for APIPA addresses (169.254.0.0/16). I could group fixed addresses for some ends, but I'm only utilizing 1 camera astatine a time, truthful I haven't been bothered to alteration it.

With amazingly small swearing astatine the SDK, isolated from immoderate complaints astir their usage of shutter clip alternatively than shutter velocity and what a "frame" is connected this camera, I put together a programme that grabbed buffers of pixels and wrote them to disk.

This was my first image retired of the camera utilizing my ain code, and I deliberation it looks beautiful bully for conscionable moving it freehand.

In bid to return it connected a train without needing to person 3 hands to clasp it, I needed a measurement to equine it to a tripod. I ended up designing a alternatively utilitarian lawsuit pinch a heat-set insert successful the bottommost that my friend Brooke 3D-printed for me. Buying the parts for it gave maine an excuse to yet make an bid from McMaster-Carr and consciousness for illustration a existent engineer.

My first effort didn't travel retired because it turns retired there's these things called "manufacturing tolerances" that I wholly forgot about.

Oops, that's a spot excessively small.

In retrospect, I astir apt should've stuck the sensors connected pinch thing different than bluish painters' tape, but it's held connected beautiful well.
Going clockwise astir it, the boards are:

6 grade of state accelerometer/gyro, which tin beryllium utilized pinch immoderate maths to to get the speed
GPS, which didn't extremity up moving arsenic good arsenic I'd hoped because the trains successful Boston are a spot excessively bully astatine blocking GPS signals
SAMD21 microcontroller to shunt the information backmost disconnected to the laptop

The lens connected the beforehand is simply a Vivitar 28mm f/2.8 that I already had for a much normal camera, pinch an adapter from Pentax K to the C-mount screw connected the camera. Since immoderate of the things I'm trying to sprout pinch it are kinda tall, its section of position worked retired beautiful well.

The full point is powered disconnected a USB-C artillery slope and there's besides ethernet and USB cables moving to my laptop, truthful it's a spot of a cablegram spaghetti monster erstwhile successful action.

With the sensors attached, I could yet springiness them a try.

Both of these images are the aforesaid seizure of waving the camera backmost and distant retired my window, but the apical 1 is the earthy image and the bottommost 1 is taking accelerometer activity into account. As you tin see, utilizing the accelerometer makes everything look a batch person to normal and little stretched. (I'll explicate much of really this useful successful a spot successful the Postprocessing Hell section.)

Once I had everything assembled, it was clip to return it connected a train.

I started disconnected connected the MBTA Orange Line, since it's the closest to me, but arsenic you tin see, the results weren't that good. Previewing what was coming retired of the camera was a pain, truthful I kinda had to conjecture connected the exposure, and I decidedly guessed wrong. The postprocessing codification I wrote didn't activity very good and everything was stretched and compressed a spot weirdly.

I went retired again connected a time pinch nicer upwind and had immoderate amended luck pinch the exposure, though I deliberation I messed up the attraction a bit. Unlike the effort pinch my telephone camera, the matter connected position signs is beautiful legible, truthful I'm decidedly getting capable lines.

I'm peculiarly happy pinch really this 1 of the Longfellow Bridge from Boston to Cambridge came out. This 1 is successful the gallery if you'd for illustration to return a person look.

When I was taking these early pictures successful Boston, I was utilizing a instrumentality from the camera vendor called Pylon to preview. The achromatic horizontal conception was each I could spot of the image astatine immoderate 1 time, and it's rotated 90° from really I'd for illustration to spot it. Dialing successful the vulnerability successful it, releasing its grip connected the camera, and past starting my ain codification backmost up earlier the train started moving again was a correct symptom that I had to do thing about.

My first effort astatine a GUI of my ain utilized OpenCV highgui, which didn't really activity for this. It requires a 1 sclerosis hold aft each frame, which is good for slower cameras, but would origin maine to miss 4 full lines (250 μs each astatine the shutter speeds I'm usually using) each show framework (256 lines).

I ended up utilizing Dear ImGUI instead, which worked nicely pinch the framework acquisition loop I already had. Out of the astir 2 twelve backends the room supports, I picked GLFW ("girl emotion for workgroups", to quote a connection from a friend astatine the time) and OpenGL3, astir apt because of the "girl love" quip, though I'm not certain.

I wrote astir of the GUI successful a azygous sleepless nighttime successful Toronto wherever rotating the image felt for illustration the azygous hardest problem successful machine science. (There's decidedly a fewer things I tin do to amended the implementation I settled on, but it runs good capable for the clip being.) Unfortunately, the pictures I took successful Toronto didn't really travel out, but astatine slightest they were exposed correctly.

I encountered immoderate unusual bugs while adding a histogram for the image.

Getting the accelerometer information proved to beryllium thing of a pain. my first type sent readings arsenic matter complete serial, which turned retired to beryllium very computationally intensive connected the microcontroller. (Converting floating constituent numbers to strings and past assembling strings is very expensive, moreover connected a comparatively powerful SAMD21 microcontroller that has thirty-two full bits.) I decided to move the conversions complete to my laptop, which has the processing powerfulness to grip them pinch ease, but this came pinch problems of its own.

A very frustrating debugging session.

The accelerometer measurements were sent arsenic earthy floating constituent numbers, but GPS information was still successful NMEA sentences and switching betwixt them required sending fixed byte sequences and hoping that thing sewage misinterpreted arsenic those sequences. (Nothing successful a NMEA condemnation should travel crossed arsenic 0x11 0x11 0x11 0x11, my accelerometer information commencement sequence, but it's not wholly intolerable for accelerometer information to incorporate 0x22 0x22 0x22 0x22, my NMEA drawstring commencement sequence.)

I besides ran into issues wherever not flushing the serial larboard astatine the correct clip ruined an full day's shots. Thankfully, I was capturing connected the Mattapan Line successful Boston, and I tin beautiful easy spell backmost and effort again.
That "seam" successful the image is wherever it mislaid each serial information for astir half a second, which is an eternity successful statement camera time. The package kept waiting for different accelerometer sample that ne'er came because the serial larboard buffer was full.

The afloat assembled camera looks for illustration a suspicious mess, and the witch utilizing it doesn't look overmuch little so.

The camera isn't usually held together pinch this overmuch tape, but I'd forgotten to bring the tripod equine sheet connected that travel to Montréal.
Would you spot her to bring unusual instrumentality onto your train?

Despite Boston's history of constabulary overreaction to harmless electronics projects, I interest the slightest astir being arrested connected the MBTA. People present thin to mind their ain business and person ne'er called the cops connected me. The constabulary besides don't thrust the trains much, preferring to harass group successful stations instead.

I'm little utilized to really things activity successful different cities, truthful I only return the camera retired erstwhile riding pinch a friend to look retired for problem (and sometimes to perceive to dispatch radio).

So far, I've only been seen, not said aliases sorted. I'm crossing my fingers that this doesn't alteration arsenic I return the camera much places.

On my travel to Montréal, I was stopped by information successful Gare Centrale and informed that tripods weren't allowed and asked, au franglais, whether I was signaling aliases taking a picture. Rather than effort to reply that philosophical mobility successful a connection I don't speak, I conscionable said "désolé" a fewer times and put distant the tripod, which seemed to beryllium sufficient.

The pictures I took successful Montréal are here successful the gallery (images 2 and 3) if you'd for illustration to spot them.

Capturing image and accelerometer information turned retired to beryllium the easy portion compared to postprocessing and making the images really look good.

The camera captured location astir 4,000 lines per second, truthful I had much lines than I needed successful each seizure and had to prime which ones really matter.

What happens if I return excessively fewer lines (Autoroute 10 successful Brossard, Québec retired of the model of the REM A)
Jumping betwixt lines excessively quickly looks artificial and wrong, for illustration is visible astatine the waterline successful this medium screen edit of an early type of the Oakland ferry photo.

To determine which lines to use, I ended up utilizing the speed, arsenic measured by an accelerometer, but this came pinch respective problems.

Firstly, accelerometers don't really measurement speed. They measurement acceleration, the complaint of alteration of velocity. By taking an integral, I tin get velocity, but that's comparative to an first value. I tin usually presume that the starting velocity is astatine a position and is frankincense zero, but I can't beryllium definite of that. If it isn't zero, I person nary bully measurement of knowing the correct worth and conscionable person to conjecture until I find 1 that smells right.

Secondly, arsenic shown successful this diagram, the accelerometer I'm utilizing is only measuring truthful quickly. The camera is grabbing lines possibly 4 times faster than it, truthful each fewer lines person to stock a velocity value. It besides isn't very accordant because my microcontroller codification isn't arsenic accelerated arsenic it could be, truthful this could origin irregularities successful the last image. How galore acceleration measurements location are aliases aren't besides changes really meticulous the integral is, which creates much problems.

You mightiness retrieve that I mentioned putting a GPS receiver connected the camera earlier, and while I did do that, it wasn't very useful. It didn't get a awesome connected astir of the trains I tried it on, and erstwhile it did negociate to get one, it only publication 10 times a second, which covers 400 full lines retired of the camera. If it worked a spot much consistently, it could beryllium useful for correcting for integration correction utilizing a Kálmán filter, but that’s a problem for erstwhile I person amended GPS data.

Even if my velocity measurement is perfect, I still person the problem of parallax, wherever things person to the camera look to move faster than things further away. This is independent of optical focus, which I usually group astatine infinity.

-u 10 region units per pixel

This problem tin beryllium dealt pinch by changing really overmuch region each pixel represents. Lower values stress things person to the camera more, while higher ones make the inheritance much visible. You tin springiness this a effort by moving the slider!

Each of these images is aforesaid size (10,000 pixels wide by 2048 tall, scaled to fresh your browser) and each includes everything from the erstwhile by kindness of covering much of the capture. The units are arbitrary and don't measurement existent region (I could make it existent meters per pixel, but I don't spot a constituent to that.)

The camera and package person nary thought what I want to "focus" on, truthful I make the creator determination and manually prime that for each conception of the image and stitch the segments together to get the pictures successful the gallery. I tested different values for region per pixel and starting velocity of each conception and past stuck them together successful GNU IMP to nutrient the last images. The assembled images often became excessively large for the 65,535x65,535 maximum size of a JPEG file, truthful I utilized the bully aged TIFF format. (The PNG specification allows likewise ample images successful theory, but the package I had to manus seems to for illustration large TIFFs amended than large PNGs.)

Notes connected region per pixel (u) and starting velocity (v) values for each portion of a fewer images

The programme that takes the accelerometer information into relationship for each statement of the image is called grindstone, since it grinds multi-gigabyte earthy captures down into smaller usable images. My first type was loosely based connected my very bad slit scan codification from earlier and was highly slow, taking hours to seizure a minutes-long capture. It would often neglect to prevention aft moving for hours because the resulting image was excessively large for the JPEG format, and debugging it was an absolute pain.

I ended up nerdsniping my friend Maddie into rewriting grindstone successful idiomatic NumPy, to make the mathematical operations that were going connected clearer (she insists that each the operations were already successful the original, and her changes were on the lines of "transforming it into a magical girl"). Maddie would later divided this type into a "perhaps somewhat overengineered" pipeline of respective different stages, making it easier to experiment, and switch successful different operations, output strategies, and the like. Thanks to her help, I've been capable to effort different combinations of parameters overmuch much easily, and get results I'm overmuch happier with.

In April, my friend Ari and I went for a thrust connected the Mattapan Line arsenic the leaves were coming successful connected the trees. The pictures I took didn't travel retired owed to a seizure package bug (see Capture) and I haven't gotten astir to going backmost yet, but it near america pinch the thought that colour statement cam photos mightiness look cool, particularly successful autumn.

While browsing ebay precocious 1 night, I recovered a very bully woody connected a colour statement camera of the aforesaid procreation arsenic the monochrome 1 I already had (the Basler ruL2098-10gc, 3x2098 pixels astatine astir 10,000 lines per second). After a spot of disassembly (it came to maine successful the lodging it was utilized successful connected immoderate mill line) and swapping the lens equine over, the camera was fresh mechanically.

I ended up putting red, green, and bluish stripes connected it truthful I could show the cameras isolated without taking the lens disconnected aliases squinting astatine mini matter connected the label.

The seizure package broadside wasn't that overmuch harder, though I did person to hole a bunch of assumptions astir the size of each statement and redo the rotation for the GUI

Progress of getting the colour seizure working

Thanks to the very modular measurement that Maddie rewrote grindstone, adding support for colour images wasn't excessively difficult, though we did person to hole immoderate strange-looking bugs.

The train was moving truthful slow and inconsistently successful this image that integration correction piled up and grindstone calculated that the camera was moving backwards and jumped to various erstwhile points successful the capture.

With capturing and processing images mostly working, much problems became apparent. The astir visible 1 is that leaves are each acold brighter than they should be.
This happens because the colour camera is delicate to infrared ray connected each 3 channels. (If it was only delicate to it connected the reddish channel, the leaves would look reddish, but the operation of each 3 channels' IR pinch the beardown visible greenish leads to the greenish achromatic successful this picture.) The monochrome camera is delicate to IR too, but it doesn't matter because it's conscionable 1 transmission and visible ray wholly drowns it out.

(Diagram taken from the camera's manual)

I will admit the effect does look beautiful bully successful the correct light. This picture taken successful Manchester-by-the-Sea, northbound of Boston, is some grayscale and colorful astatine once. (Read connected to study what the colour fringes successful the inheritance are.)

I solved this pinch an UV and IR trim select that only passes ray betwixt 400 and 700 nm, which is adjacent capable to the quality visible spectrum that everything looks right. This is the first large seizure I took pinch it, and I only needed to set the colors minimally successful post.

I besides tried a select that only passes ray longer than 720 nm (I've had quite absorbing results pinch it and IR-sensitive film), and I'm decidedly going to effort taking much pictures pinch it successful the future.

The adjacent problem is that immoderate things extremity up pinch weird red, green, and bluish fringes, particularly subjects that are further from the camera aliases moving faster.
They move retired to beryllium inherent to really this camera sensor works. Red, green, and bluish are each abstracted vertical lines (instead of a Bayer filter), and frankincense can't spot precisely the aforesaid point astatine the aforesaid time. The fringes travel from erstwhile conscionable 1 statement sees something, and it's peculiarly noticeable pinch agleam subjects. They're diagonal and not perfectly vertical because the camera itself isn't perfectly vertical. (I effort to get it close, but there's only truthful overmuch I tin do connected a moving train.)

From the camera's manual; the shaper provides formulas that tin beryllium utilized pinch the optical magnification facet of the lens and the nonstop velocity to counteract it, but I don't person (relative) velocity estimates for the subject.

I correct for it for a fixed taxable by shifting the reddish and bluish channels to statement up pinch the greenish channel. Since the lines are evenly spaced, I tin displacement by the aforesaid magnitude successful other directions alternatively than having to measurement abstracted offsets for each channel. In theory, I could determine really acold to displacement by correlating brightness shifts crossed channels, but astatine present, I do it manually.
Separation betwixt channels is visible connected the sailboat's masts, and I corrected for it by shifting the reddish transmission 10 pixels correct and the bluish transmission 10 pixels left. Color fringes are still visible successful the inheritance because it's overmuch further distant than the sailboat and frankincense has a faster angular velocity; I could displacement and correct for it, but the sailboat would look overmuch worse.

Displaying and sharing the pictures I've taken has been a symptom passim the project. Most package connected my machine doesn't for illustration really large they are, and the astir reliable instrumentality I've recovered for viewing them has been GNU IMP, which feels a spot overkill. The messaging apps I matter my friends connected get upset pinch wide images excessively and sometimes compress them into mini garbage. I was worried this symptom would proceed successful the browser, but the OpenSeadragon task had already done the dense lifting for maine and made an easy measurement to zoom astir an image.

I utilized the vips inferior to break my elephantine TIFFs up into mini JPEG tiles to service up and wrote a spot of javascript of my ain to alteration heavy links into the assemblage (mostly to make this very blog station easier). Web dev is not thing I'm peculiarly bully at, truthful I must apologize for really disfigured it ended up looking.

I person galore much ideas for this camera that I'm hoping to activity connected successful the future. The biggest 1 is to make it not limited connected a laptop to seizure images, which will make it little sketchy and easier to bring places. In bid to do so, I'll extremity up fixing immoderate of the problems that person been bothering pinch accelerometer information postulation and the seizure UI.

I'm besides readying to amended the postprocessing tools. I want to instrumentality thing that takes a spreadsheet of statement numbers and stitches and assembles from there. If I'm emotion really ambitious, I'm considering a GUI that lets maine people disconnected segments and preview them astatine different distance-per-pixel values. I'm besides tempted to effort and really usage the GPS and instrumentality a Kálmán filter, but I expect I'll put that disconnected moreover further.

Yet different point I want to effort is taking much weird infrared photos, possibly doing ærochrome-style colour swapping, for illustration what RYE does.

I besides want to qualify the mapping betwixt the summation mounting connected the camera and ISO, which would let maine to scout retired locations utilizing conscionable a ray meter. I tried to do this previously, but ray conditions extracurricular kept shifting excessively overmuch to get bully results.

The capture-side codification is disposable here and the postprocessor (grindstone) is disposable here.

I would for illustration to widen a immense information of acknowledgment to: