The Start of my Information Security Career 20 Years Ago

Update: For those who like to listen to audio instead of read, I recorded a version of this story in podcast form.

I’m feeling a bit nostalgic recently as I realized it’s been 20 years since I entered what we now call the Information Security industry. In the early to mid 1990s, I don’t necessarily know if this term existed, but viewing it through a current lens, I think it’s safe to say that’s when I entered.

Of course, if you count the couple of years of system administration I did while I was in college, I started sooner. Back then there was just the guys who ran the servers. They had to do it all, including security. Of course, in those days, particularly in academia, everything was wide open to the Internet. Then again, no one outside of academia and a few large companies even had Internet access and most people didn’t even know about it.

Regardless, in the fall of 1995, I was beginning my post-college career doing what I spent a couple years doing in college: administering various Unix and Windows systems. The company I worked for had in-house contractors they hired out to other firms. They also matched job candidates with jobs (i.e. recruiters). They eventually contracted me out as well.

My first exposure to proper hacking came in the form of a packet captures another engineer at this company had captured of a system break-in in progress and had managed to render in a way that replayed the session as if you were watching over the hacker’s shoulder while they were breaking into the system. Our company had a booth at the USENIX Conference in Monterey, CA and had intended to have it play in the background. As I was working the booth that day, and I was bored or inspired, I decided to narrate this break-in. It got a reasonable amount of attention.

I’ll be honest, I’m not sure where in this process I was exposed to a network firewall. It’s been 20 years and I don’t remember the exact chronology. The one thing I do know is that the first firewall I spent any serious time with was TIS Gauntlet.

The funny thing about Gauntlet, back in those days, was that even though it was a commercial product, you had to compile it yourself to install it. The logic behind this was sound: you should know the code that is protecting you. That said, even back in those days, the code that made up the handful of proxies that made up the TIS Gauntlet product was pretty complex. However, it made for some interesting installation failures. Also, the thought of having a compiler on your firewall probably made some people squeamish.

I also played with some other products: FWTK (or TIS Toolkit) in my home, DEC SEAL, and even ACLs on a Cisco router. But it was a fateful conversation from my boss at that time that ended up sending me towards Check Point FireWall-1: “how would you like to do tech support?”

At that point, I had no desire to do tech support. Even back then, tech support had a bad rap. Then again, as a sysadmin in a small company, I was often doing in-person tech support anyway. I decided to give it a try, even though I wasn’t sure I’d like it.

The company he sent me to contract: Qualix Group. There were two products I had to support there: Qualix HA, a server product that allowed you to run an application across multiple servers in a “highly available” fashion, and Check Point FireWall-1, which could also be used with Qualix HA. (Check Point did not have ClusterXL back in those days)

I ended up loving it, and the rest is history. I ended up doing it for more than a dozen years for three different companies, learning quite a bit about Check Point FireWall-1 and Information Security in the process. I’m glad I’m not doing that job today, but it was a great stepping stone to where I am today.

The bottom line: Don’t be afraid to try something new and outside your comfort zone. This attitude has always served me well, not only in the very beginning of my career, but throughout. No matter how it turns out, you learn something in the process that will help you take the next step in whatever direction you want to go.

Another important point: if you want to get a start in information security, work on a helpdesk or in a technical assistance center. The amount of basic, practical knowledge you’ll pick up is substantial and will help you decide exactly where in the industry where you want to focus your career.

For various reasons I decided to move the hosting of this blog to Github Pages. This also meant changing the backend to Jekyll, which I already use for phoneboy.com anyway, but I use a different hosting arrangement there. I now have a nice, spiffy new site theme as well. Not that too many people care about that stuff.

I’ve also realize in the various moves the RSS feed changed a couple of times. For those of you still using RSS feeds, it’s now: http://phoneboy.org/index.xml

From Palo Alto Networks: Best Practices for Securing Your Network from Layer 4 and Layer 7 Evasions:

To monitor and protect your network from most Layer 4 and Layer 7 attacks, here are a few recommendations

While I’m all about Best Practices documentation, one wonders why some of the items in this documentation are even required.

Block all unknown applications/traffic using security policy. Typically, the only applications that are classified as unknown traffic are internal or custom applications on your network, or potential threats.

Wouldn’t a better approach be to only permit the applications you want to allow rather than to block all unknown applications? Oh, wait, that’s not how that product was designed to work…

Create a zone protection profile that is configured to protect against packet-based attacks: * Remove TCP timestamps on SYN packets before the firewall forwards the packet […] * Drop malformed packets. * Drop mismatched and overlapping TCP segments

A proper stateful firewall should do all of these things by default, shouldn’t it? Same with the half dozen other checks they say to enable via the CLI?

Doesn’t it seem strange to anyone else that you have to manually configure so many options to make the device more secure? What’s the likelihood customers have actually deployed their gateways in this fashion? Are they getting the performance they expect if they do so? My guess: probably not.

To add a little more fuel to the fire: Firestorm. This attack uses TCP SYN + Data packets to bypass certain firewalls. Per the article:

We disclosed the full details of the vulnerability to major vendors affected by the flaw. One of the vendors who replied, explained that they do not see this issue as a vulnerability because, by design, their firewall permits full TCP handshake in order to inspect the application type.

They said that once their state machine proceeded beyond the TCP handshake, they would recognize the application, matching a subsequent rule that applied to application traffic. The vendor added that if there was an application they did not recognize, they would treat the session as ‘unknown-TCP’ and, again, perform an additional security policy lookup to decide whether to allow or block the traffic.

Now that this information is out there, it is only a matter of time before some develops a tool that can be used to bypass these security devices and, short of blocking all traffic, there’s nothing you can do to mitigate the threat “by design.” Oh wait, someone already has. (Note: There was a tool at https://github.com/stasvolf/SynTunnel but it has been removed)

It should be noted that Check Point Security Gateways are not susceptible to Firestorm.

Disclaimer: While I work for Check Point Software (a competitor to Palo Alto Networks), the above thoughts are my own.

There have been a few videos produced that show various ways to bypass Palo Alto Networks firewalls. This is the latest, complete with a configuration file and a pastebin log from the Evader tool showing the various exploits that were triggered:

I don’t know enough to evaluate the claim made in this video that these flaws are fundamental to the architecture of the Palo Alto Networks gateways. I do know that Palo Alto Networks disputed this video privately, and a response to it was recorded, showing the same issues as before. If the video is factually incorrect, why hasn’t Palo Alto Networks posted a public, formal response via their website, YouTube, or social media? The fact they haven’t, make of that what you will, but when challenged on similar issues in the past, they first denied it and later they recanted.

I wonder: how do organizations who purchase this product decide a particular product meets their needs? Are organizations doing a true evaluation pitting a number of security tools against a set of objectively-defined criteria or did a decision maker somewhere get wowed by the marketing and bought it without a serious evaluation?

Based on many of the request for proposals and proof of concepts I’ve been involved with, more often than not, it seems to be the latter a lot more often than the former.

Check Point CEO Gil Shwed said during the Q3 2015 earnings call: “We should work harder to expose the difference between marketing hype and technology that actually works.”

The best way to protect yourself from the marketing hype is to understand what your actual security needs are, define objective evaluation criteria, and put the tools through their paces to see which ones is best for you.

Disclaimer: I work for Check Point Software Technologies, which is a competitor of Palo Alto Networks. The views herein are my own.

It’s very easy to get discouraged in the information security business. Every piece of software, every software as a service we use is potentially vulnerable to security threats: some known, many likely not known. When these threats are exploited–it’s no longer a question of if–data and reputation loss are likely results. Even if you’re secured the central repositories of this data, the client devices that access that data, perhaps even storing that data, have their own vulnerabilities and threats. When you sprinkle in configuration errors that are all too prevalent and permit more access to resources than absolutely required, it’s easy to come to the conclusion that the game is over, the jig is up, we’re compromised, and we’re done.

The worst of all this is: you most likely don’t even know what resources you have. Even when you know, you probably don’t have a lot of say into who can access what resource how. When you try to bring this to the attention of the executives to get more resources to address the issues, the executives don’t see the value.

Over the last couple of years, I’ve been working with Check Point customers to understand their specific situations and come up with a long-term game plan. As part of that process, I try to find out what’s truly important at the business level. This means not talking to the technical people, but to the business leaders. This helps provide some clarity on what of the thousands of potential security issues out there needs the greatest focus.

It’s also important to enumerate what’s in the environment, starting with the critical assets. Where are they? Who accesses them? What security controls are in place to ensure only authorized persons can access those resources in a non-malicious way? A logical network diagram showing where everything is and understanding the various traffic flows is very helpful in figuring this out.

The presence of controls in the environment is one thing. Are they configured to per the principle of least privilege? Are those controls logging? Are you actually reading those logs and/or using a properly Security Information and Event Management product to help contextualize what’s happening? Are you acting on the information these tools are giving you? If a serious breach does occur, do you have a plan in place?

I’m sure there are a lot more questions I could ask (and sometimes do, depending on the customer). However, there is only so much information I can gather over the course of two or three days. I then take this information and write a report with recommendations. These reports can be somewhat long, depending on the customer.

What I’ve also started doing, which I believe is more valuable, is summarizing all the relevant information in a spreadsheet. It’s designed to be executive friendly, showing the issues, relative risks (with color codes), recommendations, cost to improve, and so on. It’s by no means perfect, but the goal is to bring a bit of order to the chaos–showing a potential plan to move forward and a framework you can use to re-evaluate the situation in the future.

The question I ask of my fellow information security professionals: how are you helping your organization bring order to the chaos of Information Security? Are you just reacting to events as they occur–something that is unavoidable–or do you have a long-term strategy in place that you are actively implementing?