A Eulogy for Raspi03

I’ve had Raspberry Pi models 2&3 in continuous commercial service for five years. Transition to cloud and local virtual machines eliminated some of the need; COVID-19 has been eliminating the rest as offices close. These are amazing and capable devices; I’m sure they’ll be redeployed again soon 🙂

If you’re interested in “Guerrilla Networking” ~ making the most out of your budget, minimizing your physical footprint on the network, minimizing power consumption, etc. ~ or developing your skills with a minimal investment ~ consider picking up a few Raspberry Pis and build your skills! For guidance along the way ~ or just to have it done ~ you know where to find us: Contact!

Without a flourish, and without any fanfare, albeit with a little bit of reverence, I pulled the plug and it was done: Around five years after first being put into service for monitoring and maintaining client networks, “raspi03” was taken out of service. Those comforting little green and red LED’s visible through the clear case went dark; its pulse flat-lined on the switch. It was… done.

Raspi03 was the third in a collection of four Raspbery Pi (Gen. 2) that I used for different sensor and networking experiments before I could create a proper VM server. When a client found itself stuck with a particular set of requirements surrounding remote maintenance and monitoring, raspi03 stayed on the home network while raspi02 and raspi04 deployed forward onto client networks.

Raspi03 initially served as an OpenVPN server while 02 & 04 called home. 02 & 04 served initially as “jump boxes” of sorts. I built them out to hold a wiki for the clients on site and for admins needing technical details. Soon enough, 02 & 04 also ran help desk software for submitting trouble tickets, tftp servers for collecting local running-configs from client infrastructure devices, syslog service to collect infrastructure logs, nagios to monitor devices and network links, ARP monitoring, WiFi environment scanning, and so forth.

Raspi03, as the OpenVPN server and de facto network hub, also ran nagios for the “outside looking in” view of the various sites, a mail server for handling inbound alerts from site, DNS, syslog again to receive consolidated data from sites, and more. Access controls were tight with low level tools such as hand-crafted iptables, NAT, and routing rules. Inbound, outbound, client nets, VPN, and internet were all guarded. The VPN evolved to isolate site devices from client/administrator/user accesses. Where appropriate, the VPN allowed routing from headquarters directly through to client devices.

Raspi03 evolved to run Kerberos and LDAP services for user and system account access controls, limiting particular administrators to particular sites and services. Site web accesses evolved from open, to simple user/password schemes, to LDAP-based controls, and eventually to OpenID-connect controls.

While security scans, for instance from OpenVAS or similar, did not originate on a Raspberry Pi, the scans were pumped through the VPN connections and through the site Raspberry Pi’s to keep tabs on vulnerabilities. On-site encrypted volumes held change logs, scripts, and data for performing periodic maintenance, such as backups, password changes, and so forth.

Raspi03 also served as the first gateway for clients to access my lab networks, where I could build out and demonstrate the proofs of concept that they would want to implement in their own networks. We had wikis, IRC channels, and even site-to-site private telephony routing through the Raspberry Pis. During security engagements with social engineering, network attacks, infiltration, and exfiltration, data was inevitably routed through raspi03 as well. Where appropriate, a client might stand up a virtual machine as a remote point of presence for me on their networks, eliminating the need for a deployed pi in such cases. Additionally, I’d also duplicate those basic local functions in Amazon AWS instances and local virtual machines, but raspi03 and the “overwatch.vpn” were inevitably in the background as fail-overs where appropriate.

By the way, raspi01 ~ a first generation Raspberry Pi ~ had an attached camera, motion sensor, and temp/humidity sensor. It was occasionally put into service monitoring server closets, sending back motion-triggered photos and environmental telemetry. The capabilities of the devices, even from the first generation, remain astounding.

Last week, raspi04, the last of the forward-deployed pi’s was decommissioned. Today, venerable raspi03, the original network hub for it all, was decommissioned ~ all network couplings were removed from the office routing, and all services were shut down or transitioned to other devices or instances. That ended five years of raspberry pi’s at the edge and in the core ~ continuous operations, packed with functionality, providing a bespoke, secure, and professional presence. It was a lot different than the big budget folks just throwing up a high-dollar network gear and cloud images on a whim ~ more along the lines of “guerrilla networking”… It was certainly a wild ride.

Maybe the information is worth preserving and teaching for the non-profits and budget-minded ~ who knows? Working with the low budget folks in “Basic IT” isn’t a great way to feed the family though… It’s bittersweet that some efforts are closing ~ think COVID-19 and idle office networks… I am curious where we’ll head next.

… and I am curious how I’ll inevitably reconfigure and deploy raspi03 next. It’s just a flash of the microSD card ~ or maybe a fresh one after all these years ~ and we’re ready to begin again.

Raspi03 is Dead! Long live Raspi03!

“This is Why Half the Internet Shut Down Today”

Spoiler Alert: Cloudflare DNS.

According to the incident report, this issue with Cloudflare’s DNS service impacted its data centers internationally, from Frankfurt to Paris and Schiphol, as well as several in major U.S. cities, including Los Angeles, Chicago, Seattle, Atlanta, and San Jose. Reports on Downdetector showed the outages appeared to be concentrated in the U.S. and northern Europe.

Gizmodo, yesterday (link)

As a reminder, Firefox has been rolling out their DNS over HTTPS service with Cloudflare as the default provider. From Mozilla in February:

We’re enabling DoH by default only in the US. If you’re outside of the US and would like to enable DoH, you’re welcome to do so by going to Settings, then General, then scroll down to Networking Settings and click the Settings button on the right. Here you can enable DNS over HTTPS by clicking, and a checkbox will appear. By default, this change will send your encrypted DNS requests to Cloudflare.

Mozilla’s Blog (link)

If you’re a regular reader, you’re probably accustomed to seeing a cautionary post on DNS on the front page here. Really, there’s no shortage — most recently:

There are also quite a few citing DNS within them, and undoubtedly there will be more. So much of the general function of the internet and internal networks depends on the DNS underpinning — to include both ordinary and security functions. At the most common level, if DNS is down, you don’t find “google.com”. At the most sophisticated, you’re sent to a very tailored “google.com”. In between, ads are blocked, hostile websites are sinkholed, email servers are validated, LDAP & Kerberos servers are discovered, corporate users are taken to internal, privileged versions of websites, and so forth.

DNS is serious business. It’s hard enough to keep control of your own network operations without users and services bypassing your controls; it’s another thing altogether when those bypasses take you to compromised, hostile, or faulty services.

Stay vigilant, and stay in touch! Find our Contact form here.

OBS & KDENLIVE Quickstarts

Part 1: OBS Overview
Part 2: A quick Kdenlive edit

If your organization lacks a formal PR Department, lacks an in-house capability to produce training videos or instructional HOWTO’s, or you would just like the ability to bootstrap some self-promotion through your website or social media, exploring OBS & Kdenlive — two free and open source software projects — might be worthwhile.

I’m not going to lie: It is a challenge ~ but with practice, the skills improve. Maybe results will follow 🙂