Q&A:

You've been posting a lot about this firm Grancius. I assume they've left a lot of data exposed. I also assume this exposure is important.

Yes and yes. Granicus delivers an enormous amount of software / IT services / hosting infrastructure to governments in the United States and the United Kingdom. Also, due to their private equity owner and some mergers that happened a few years ago, they aren't just Granicus. They are: GovDelivery, GovLoop, GovInteract, and Acquia to name a few. Acquia supports Drupal by the way, which is very similar to WordPress. Whitehouse.gov runs on Drupal. As for GovInteract, these people run part of the national Emergency Alert System for FEMA and the Department of Homeland Security. This is serious stuff.

Ok so they're important and a major data leak or breach or whatever you're calling it would be bad. But isn't all government IT bad anyway? Also does this data even impact the sensitive parts you mentioned, (like the ENS system) or are you making dubious correlations?

For about 18 months I worked with pretty much all State of Texas government agencies. There are almost 40 of them.  Government IT is old and runs slow, but surprisingly it is usually well architected and done to spec, even if that spec is on an IBM z-series mainframe. Despite a very limited IT budget in Texas, I never saw an agency approach the level of misconfiguration seen here. It is *weird*, and Granicus needs to have a come-to-Jesus-moment and to figure out what happened.

Also, there is no direct connection to the ENS system, if we are talking about the so-called Presidential Alert system for national emergencies. It is likely that similar people manage both systems, as I'll describe here. It's enough to be concerned. That being said, there are still incredible ways to deliver malware into Granicus itself, and the damage that would cause makes the ENS system the least of our worries.

Okay, what happened?

They left numerous servers open to the public internet that allow customer data to be modified and/or deleted by anyone. Doing this requires just an iPhone and typing out a couple of commands. These aren't hypothetical exploits that people drum up press for (I usually find those serious as well, but I will admit it is hard to feel that way if you aren't deeply involved with the technology). It would not be inconceivable for a child messing around on their parents iPhone to modify one of these databases it is so simple at times.

These servers support around 1700 Granicus government customers in the United States. Taking them offline would take a portion of many local, state, and Federal agencies offline. Sometimes, this means all video and audio on the site wouldn't work (the links would return a 404 error). In other cases, it means the sites would go offline completely. In all cases, it would be trivial to replace links on production US government websites, such as https://senate.gov, https://armedservices.congress.gov, and https://amarillo.granicus.com/, to instead download incredibly effective malware instead of the expected audio and video files.

There's no way they left a major database open!

Screenshot of the major database that Granicus left open.


Okay, but there's no way you can add / write / manipulate data on it!

See https://notes.12security.com/2020/02/granicus-essay-3-etheric-networks.html

Okay fine, but the swapping in malware part is totally not true.

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html

Ugh. Well everything you say is true, but no one is going to do anything anyway. Stop making this my issue since you know I'm an equally powerless reader.

Ahh, but here is where things get interesting and we come to a much debated legal term, "prior restraint". First, you should know I have uploaded code to one of the major cloud providers and their FaaS service. In 91 days, this code will swap out all URLs on this production server with something else.

I don't want to get into too much detail here, but I have gone and will continue to go to great lengths to alert Granicus, even down to emailing individual employees. This will also include the exact code fix they need to apply and how it can be done in likely less than 10 minutes. I've actually written multiple code scripts because I can't be 100% sure of the backend issues and I want their architecture to still work perfectly with zero impact.

Sadly I'm not still sure that despite all this Granicus will act. I think it is very likely executives of a modern day corporate leadership would rather let the firm fail and cease to exist than change anything tangible. I know it sounds preposterous, but that was only a conclusion I came to after seeing over and over again this weird executive paralysis when confronted with cyber issues or anything related to the common good.

NOTE: Actually I wound up contacting NIST, (a Granicus customer!) and to the best of my knowledge they alerted Granicus + a few intrepid reporters + a very super intrepid reporter and caused them to act. Even if this function were still up, it would no longer work as the underlying issue has been fixed.

What is the other weird stuff you mentioned with Granicus?

I've talked about that a little in other posts and will detail that more in the future. But suffice to say, it is not just this open database. There are other things. And together it is very bad, very scary, and I really hope it is all just gross negligence. I don't know what comes after gross negligence, and I understand sometimes historical architecture choices and legacy IT leads to what looks like delusional choices that actually make quite a bit of sense. That being said I actually have a huge interest in computer history, work to restore decades old systems in my spare time, and have visited often the only two computer museums in the US (CHM in Mountain View and Paul Allen's wonderful small museum in South Seattle). Yet despite all of that I can't think of any legacy IT reason that would leave one to make their production database write-open to the world.

What if you're wrong here though? Wow, that would be embarrassing.

I had to think a lot about this. This includes the notion of what being right means, and the likelihood others have noticed similar issues and not spoken up out of fear or because they are without some obscure technical knowledge. I had to think about whether more disasters been caused by lack of voices being raised, or being caused by false alarms; and if I am "wrong", whether I'm both able and willing to, in good faith, bear the consequences. In the end, I hope I am wrong. Because if I am right, there are 20 other similar instances in the back of my mind of similar companies with similar set ups and responsibility to Granicus I've seen in the last 12 months. That idea makes my fingers shake above the keyboard and I don't want to dwell on it again until I absolutely have to.