The Story of a WebApp #devsecops?

All of the sudden, it’s not looking much like a coder’s boot camp exercise anymore…

I’m certain I’ve told the story before, but the email reminder from my carrier to pay the family cellphone bill arrived last night while I was tweaking the code… Yeah, it’s time to recap 🙂

The Backstory

All the cool kids have their own cellphones, right? Even the non-mobile ones, it seems. My son has muscular dystrophy and doesn’t get out much. When he does, he’s generally not far from someone with a cellphone. So, does he really need one? In his own home environment, he has a laptop and an android tablet with whatever apps he wants. Additionally, he has an office desk phone with his own phone number, house-internal extension, and voicemail. What could possibly be missing?

“Text messaging, Duh!”

But is text messaging worth another $800 +/- device plus the crazy recurring phone company fees?

The Exercise

So, I mentioned the office desk phone, yes? Given that we already pay for internet access, voice over IP (VoIP) telephony was a fairly easy add. Well, sort of easy. Here, I don’t mean adding on to the ISP’s plan or calling some other carrier; rather, I mean we run our own FreePBX server in the house and adding a phone number from a SIP provider amounts to around $1.25 per month.

It turns out that that SIP provider also maintains an API for SMS & MMS sending, receiving, and tracking. For all practical purposes, there’s no appreciable additional cost for using it. I’ve used the API directly to send different system alerts to my own cellphone, and I’ve even set up simple SMS-to-email for inbound contacts for clients who would inevitably try to text message one of the office lines… What I hadn’t done was to strap it all together in a convenient package.

The Iceberg Principle

This is the potentially demoralizing part. Today a simple messaging application might be a standard computer science class or coding boot camp exercise. That’s the visible part. From the perspective of systems architects and security professionals, the more interesting details are below the surface. Plus, to test the actual depth of your understanding each step of the way, you get to build out the underlying infrastructure as well — jobs ordinarily handed out to different teams: infrastructure, security, application engineering (back-end, front-end UI/UX), etc.

Solution, Mark I

Before the diagrams, videos, and whatever else may follow, let’s start with a casual rough-out of the basic requirements and the resulting design.

  1. We’re going to need a fixed IP address to receive inbound messages and a DNS-resolvable name for easy access to the service from the internet at large. The initial service, however, will run on a self-hosted server on a network behind a dynamic IP address. HTTPS (SSL) is an obvious need.
  2. The SMS / MMS service is going to handle multiple users and multiple phone numbers, with different users having access to different sets of numbers.
  3. While I can write a simple web front end for myself and my son, more skilled front end developers can undoubtedly put a better wrapper around the back end services. Additionally, other system services are expected to use local SMS service — for instance, for system alerts and for interaction with users. It makes sense, then, that we should have a RESTful back end.
  4. Since different activities, such as emailing a copy of an inbound message to subscribed users, do not require that a user be logged in, there should be databasing for message storage and an event queue of sorts that side-services can monitor.
  5. For a while this will be a low demand, low bandwidth service — nothing critical, no high availability requirements, etc. However, we should be mindful of bad actors, particularly while opening access to the internet.

So, how do we sketch out a solution?

  1. The Forward Edge
    1. We stand up an Amazon cloud EC2 micro server with a static (“elastic”) IP address, running Ubuntu 18 (~$10/month). We already have the domain name and manage the DNS records through that provider, so we just need to add the A or C record as appropriate.
    2. This “box” will receive HTTP/HTTPS traffic and proxy it back to the lab. For that, we’ll run nginx for reverse-proxying and we’ll run certbot to generate basic SSL certificates, both services strapped together with docker-compose for initial convenience. With the nginx configured right and DNS set correctly, the SSL registration and deployment is a fairly easy operation.
    3. We add the box to a locked-down OpenVPN network client. (The office network behind a dynamic IP address has an Ubiquiti ER-X at the edge, which is also a client of that same OpenVPN network, so we can construct a path back.)
    4. We join the cloud box to the local, internal FreeIPA realm for enhanced access controls and maintenance, adding a few lock-down tweaks in the process.
    5. We add it to the lab’s Nagios monitoring to track for basic issues and alert if something goes off the rails.
  2. Authentication & Authorization in the Middle
    1. We add a Keycloak test realm specifically for this project. Keycloak will handle the test user management, password complexity management, two-factor authentication (TOTP 2FA), and assignment of telephones for users and services accounts.
  3. The Back End RESTful Service
    1. We use Flask (a python micro-services framework) for the back-end web application routing and functions.
    2. We use REDIS for the alert bus with its publish/subscribe queues.
    3. RESTful endpoints use the OpenID-Connect protocol, relying on Keycloak as an identity provider for authentication as well as authorization claims containing the user’s provisioned phone numbers.
  4. A Simple Front End Prototype
    1. We construct a simple javascript single-page application for live sending and receiving of SMS & MMS messages, relying on server-side event (SSE) streams (attached to the REDIS queues) for updates and AJAX calls to the RESTful endpoints for sending messages.
  5. Security Concerns
    1. Given the architecture, we walk through the system from the edge through the back end looking for security weaknesses and constructing mitigations.
    2. (We have a particular example of the SIP vendor’s use of Amazon Lambda services involved in MMS attachment pick-up that cannot guarantee fixed IP source addresses for whitelisting. That requires us to hold open the door to the public waiting for the callback…)
    3. We walk through network design, VLAN isolations, and firewall rules used to limit access and to contain damage.
    4. We walk through access controls for the servers, backups of data and virtual machine shap shots, detecting break-ins and tampering, and so forth.
    5. Etc.

The Bottom Line?

All of the sudden, it’s not looking much like a coder’s boot camp exercise anymore… Application developers, full-stack developers, systems engineers, infrastructure engineers, systems administrators, and others, all have their own perspectives and are not too heavily burdened by subject matter expertise of the others.

Security — good security — is holistic, requiring significant depth and experience across the board to see the system as a whole and to identify areas of concern.

Security — the best security — also finds a way to mitigate those risks so the application still gets deployed and business meets its objectives. These security folks are assets that help move the business forward.

Interested in learning more?

The tip of the iceberg, that javascript UI? It’s not very interesting; my javascript & css fu is not very strong. The rest though? That is particularly interesting to me as a longtime systems engineer / systems integrator / security professional. If it’s interesting to you as well, let me know and we can work through various aspects as a series.

In the meantime, drop us a note via our contact form if a business-focused approach using broad spectrum security might benefit you.