Back when I worked for the Nokia TAC support Check Point Security Gateways, I actually ran a Nokia appliance as the headend for my network. One could call it "keeping my skills sharp for my daily work." For a variety of reasons, I discontinued this practice and instead opted for more traditional home routers that I could install DD-WRT on.

When I started working for Check Point, I alternated between either running a DD-WRT router or using a UTM-1 EDGE appliance. When Check Point announced the SG80 (and later the 1100 series appliance), I moved to using those. 

And while I probably could continue to use the 1100 series appliance for some time, I finally got my hands on a 2200 Appliance and decided that it should be the headend to my home Internet connection. I wanted to be able to use all the latest Check Point features and get my hands a bit dirtier than I have in a while. What better way to do that than with my family, which is very good at generating lots of real-world, Internet-bound traffic :)

DHCP and DNS Server on the Gateway

Every gateway I've used in the last several years had a DHCP server. In most cases, it also included a DNS server. Gaia does have a built-in DHCP server, and should you use it, should make sure to configure your rulebase to permit the traffic correctly as described in sk98839.

That said, it's not currently possible to make the Gaia DHCP server authoritative for a given subnet without manually hacking the dhcpd.conf file (see sk92436 and sk101525). I did this, which worked.

I still wanted a local DNS server too where I could define local DNS entries for items on my subnet. I could have easily used dnsmasq on one of the many DD-WRT routers I still have on my network as WiFi access points, but I wanted it on the Security Gateway, just like I had it on my 1100, UTM-1 EDGE, and others. I poked around a bit on my Gaia R77.20 installation and discovered that it too had dnsmasq.

I want to be clear: dnsmasq is not formally supported on Gaia. That said, it appears to work quite nicely. The configuration file /etc/dnsmasq.conf is not managed through Gaia and, as far as I know, it should survive upgrades and the like and not be affected by anything you do in Gaia, except maybe enable the DHCP server in Gaia, which obviously do you don't want to do if you're going to use dnsmasq for this.

The dnsmasq in Gaia appears to take most of the common configuration options, so I set about configuring it as described in the dnsmasq documentation. Starting it up is pretty simple: a service dnsmasq start from Expert mode gets it going.

But, of course, that only works until the next reboot. Now I could have hacked /etc/rc.local or something like that, but I know how to make Gaia's configuration system actually start this process for me, monitor it, and restart it if it dies.  

[[email protected]:0]# dbset process:dnsmasq t
[[email protected]:0]# dbset process:dnsmasq:path /usr/sbin
[[email protected]:0]# dbset process:dnsmasq:runlevel 3
[[email protected]:0]# dbset :save

I guess all those years of supporting IPSO paid off :) If you want to turn off dnsmasq in the future, issue the following commands from Expert mode:

[[email protected]:0]# dbset process:dnsmasq
[[email protected]:0]# dbset :save

One other thing you might want to do if you're running a local DNS server is have your Security Gateway actually use it. The problem is: dhclient (which obtains settings from a DHCP server) actually overwrites the configuration in Gaia for the DNS domain and the DNS servers to use. You should be able to prevent this by changing the /etc/dhclient.conf configuration file not to use this information.

Specifically, it means changing this line:

request ntp-servers, subnet-mask, host-name, routers, domain-name-servers, domain-name;

to this:

request ntp-servers, subnet-mask, host-name, routers;

To be absolutely clear, Check Point does not support using dnsmasq at all for any purpose on Gaia. While I am using it successfully in R77.20, it could easily be removed in a future version at a future date.

DHCP and the Default Route on the Internet Interface

When I put the 2200 in my network, I noticed that Gaia was not accepting the default route from Comcast's DHCP server. To get my home network oeprational, I manually set the default route to my Internet-facing interface, which seemed to work for a time.

One persistent issue I was having, that I initially was unable to figure out, was that it would take some time in order for sites to load. I wasn't quite sure why, but it was only happening occasionally and not something I could easily track down. I ignored it for a couple of weeks, but last night the problem really came to a head.

I thought Comcast was having a problem. I even went so far as to call Comcast and they said there was some signal problem on their end. The problem still persisted. The only thing "different" was that we had visitors who brought a laptop with him and was playing League of Legends with my son.

I tried a number of things, and couldn't figure out what was going on. I did start seeing some peculiar traffic on my external interface, and decided on a fluke to check my arp table. Rather than the 20-30 entries I'm used to seeing, I was seeing over 700 entries. For IPs on the Internet. That didn't look right. And sure enough, I was seeing arp whois requests for Internet IPs coming from my gateway.

Arping for every IP you access on the Internet is not terribly efficient. Comcast was, understandably, not responding to these ARP requests quickly, though they eventually did. Furthermore, due to the transient nature of ARP entries, they were timing out every minute or so, which meant that as long as I was actively passing traffic, connections to websites worked great. When I stopped and reconnected? It would take forever to connect.

Which brings me back to the original problem I had: getting Gaia to take a default route from a DHCP server. Turns out, there's an option in Gaia called "kernel routes" that you need to enable to make this work, and enabling it is pretty straightforward. If I had search SecureKnowledge earlier, I probably could have saved myself learning (and documenting) this lesson. It is documented in sk95869.

Now the Internet at home is working great once  No delays in loading websites at all. At least ones caused by configuration errors on my part :)

Dynamic IP Gateway Security Policy

One of the challenges of using a Dynamic IP Gateway is that you don't know what the external IP is of your security gateway. Not knowing it could make it difficult to, say, make "port forwarding" rules or rules that relate to allowing traffic to the Security Gateway itself.

Turns out this is not an issue. When you check a gateway object as Dynamic IP as shown below, two Dynamic Objects come into play: LocalMachine (which always resolves to your gateway's external IP address) and LocalMachine_All_Interfaces which resolves to all of the IP addresses associated with your gateway. These dynamic objects are simply placeholder objects that get their actual definition updated on the local gateway. For these two objects in particular, this happens periodically without any action on the administrator's part.

Dynamic Objects have a couple of important limitations. Rules using Dynamic Objects must be installed on a specific installation target, i.e. you cannot simply leave the Install On field for the rule the default Policy Targets. Not doing so will cause a policy compilation error. The second issue is that rules with Dynamic Objects disable SecureXL termplates at that rule. This isn't a huge deal as I'm coming nowhere near the capacity of my 2200, and most of my traffic is hitting rules with acceleration, but it is a consideration you must make when constructing your rulebase. 

Another important limitation: Identity Awareness is not supported on Security Gateways that have a Dynamic IP. I do not believe this is an issue on the 600/1100 appliances, but it is for the 2200 and up. 

Final Thoughts

It's been a while since I posted something technical on my own blog about using Check Point. My role is such that I'm not always involved in the technical side of things like I was back when I wasn't quite Internet famous :)

In any case, it may be something I'll do again in the future. I'm sure that pieces of this article will also show up in SecureKnowledge--the supported parts of the above at least.

And, of course, these are my own thoughts and experiences. Corrections and feedback are welcome.

A blog post written by Greg Ferro, a.k.a. EtherealMind, had come to my attention on a review of ClusterXL in Check Point R77 based on the documentation. I'm familiar with Ferro from the Packet Pushers Podcast where he and Ethan Banks talk nerdy about networking. They occasionally dip into the security arena and when Check Point does come up, it's not in a positive light.

When I actually read Ferro's post about ClusterXL, I was not surprised that the end result was a negative review: not only of ClusterXL but of Check Point itself. What surprised me was how incomplete the review was, technically speaking, because I know how nerdy he can get.

Ferro only used one document for his paper review: the Check Point R77 Versions ClusterXL Admin Guide dated 28 July 2014. He could have easily asked the customer he supposedly did this review for to obtain additional documentation, such as the ClusterXL Advanced Technical Reference Guide in sk93306. Because he didn't have direct access to said documentation as an independent consultant, he chose not to use it.

What is ClusterXL anyway?

Even with the documentation Ferro chose to use, the more I read his Tech Notes Check Point Firewall ClusterXL in 2014 piece, the more I conclude his review was not very thorough. Clearly he read some of the documentation as there are quotes from it throughout the piece, but it's obvious he missed the part that defined what ClusterXL actually is:

ClusterXL is a Check Point software-based cluster solution for Security Gateway redundancy and Load Sharing. A ClusterXL Security Cluster contains identical Check Point Security Gateways.

  • A High Availability Security Cluster ensures Security Gateway and VPN connection redundancy by providing transparent failover to a backup Security Gateway in the event of failure.
  • A Load Sharing Security Cluster provides reliability and also increases performance, as all members are active

Ferro's definition of ClusterXL?

There seems to be two modes of active/active High Availability for CheckPoint Firewall one. Both of which are called ClusterXL.

Ferro then talks about the requirements for ClusterXL, which he gets mostly right: sync is a Layer 2 protocol, ClusterXL requires appropriate licenses (which every current Check Point appliance has, as do many current software-only firewall SKUs), and you should use NTP between members. Actually NTP is a good idea for reasons other than state sync (e.g. VPNs and SSL).

State Synchronization

Then Ferro comments on State Synchronization.

  • by defaults all connection state is synchronised across the cluster.
  • you can decide not to synchronise. Why ? Does this imply a performance issue in the devices, network, speed or other issues ? What would I not sync all state ?
  • “Synchronization incurs a performance cost” – implies that CheckPoint performance remains a problem for “many customers

For the vast majority of Check Point customers, state sync does not have performance issues. Where issues have been observed is in environments where the connection rate is far higher than the throughput would suggest. You can reduce the load by disabling sync for services with connections that are transient in nature (e.g. DNS, short-lived HTTP connections) and can be done easily in SmartDashboard.

But that's not all he has to say about State Synchronization:

  • Synchronisation Network requires elevated security profile, suggests CCP is insecure protocol, isolated network.
  • Recommends using an isolated switch – this is stupidly impractical who the heck wants to waste money and power on a dedicated switch in the DMZ.
  • Using a crossover cable is equally stupid and impractical. What are they thinking ?
  • Appears that sync is ONLY supported on the lowest VLAN ID on an interface.

The Synchronization Network requires an elevated security profile: yes, this is stated in the documentation and is no different than every other major vendor. A dedicated switch or a crossover cable allows this elevated security profile to be maintained better than a dedicated VLAN, which by the way, if you want to use a dedicated VLAN, yes, this is supported; it's a common configuration in customer networks. As to whether a dedicated switch or crossover cable is "stupidly impractical," customers have asked for support for all of these options, which is why they are tested and supported options (and thus listed in the documentation as such).

Ferro's next complaint? You can't use more than one synchronization interface. This actually used to be a supported configuration once upon a time but in recent versions, the supported approach is to let the operating system handle interface redundancy via bonded interfaces (i.e. with LACP). I asked in the comments of the post why he felt using LACP was an inferior solution to multiple independent synchronization interfaces. No response.

ClusterXL Load Sharing

Ferro never once explicitly mentions the sort of configuration he was using as the basis for his paper review of ClusterXL. Based on the fact he only briefly mentions High Availability (or Active/Passive, as he said) and spends the majority of his time talking about Load Sharing (Active/Active), I can only assume that is the configuration he's most interested in, ignoring the fact that High Availability configurations are the far more common customer configuration.

Of the two ClusterXL modes that relate to Load Sharing, Ferro only talks about one: Load Sharing Multicast. This mode, according to Ferro, "should be avoided at all costs" because it involves implementing workarounds that are, in his words, "complex and hard to maintain/operate." These workarounds, static multicast mappings and/or disabling IGMP, are necessary because some networking equipment out there does not properly support RFCs or has issues with frame replication (needed for Multicast). 

ClusterXL has a mode that requires none of these workarounds: Load Sharing Unicast mode. This mode was only mentioned once, but no analysis of this mode was offered by Ferro.

Ferro's opinion about VPN and Load Sharing is that "there are many conditional statements about VPN connections" and "would have little confidence in running Load Sharing Multicast Mode and running VPN services on the same physical unit." I couldn't find the conditional statements he was referring to, but what I did find was one particular statement in relation to third party peers and load sharing (repeated in a couple of different ways):

The third-party peers, lacking the ability to store more than one set of SAs, cannot negotiate a VPN tunnel with multiple cluster members, and therefore the cluster member cannot complete the routing transaction.

The problem isn't Check Point, per-se, the problem is with interoperating with non-Check Point devices. The workaround: using Sticky Decision Function, which is listed in the documentation where this "conditional" statement is listed. This, along with the other so-called conditional statements he refers to are curiously absent from his paper review.

Performance and Cost

Ferro writes in several digs at Check Point's allegedly "punishingly expensive" products. In most of the competitive situations I've been engaged with during my tenure at Check Point, pricing compared to the competition is rarely a factor. That said, I have seen numerous situations where competitors have purposely specified lower-end (and thus cheaper) solutions that, in reality, will not meet the customer's stated requirements while maintaining the same security posture.

Ferro also states "forwarding performance is and price/performance is very poor" which is provably false. Independent third parties have verified Check Point's performance and pricing and have found Check Point in-line with the competition. It's also something being validated internally at Check Point on an ongoing basis.

Ferro is correct that performance decreases when additional security features are enabled, however this phenomenon is by no means unique to Check Point products. The expected performance for a given Check Point appliance is listed right on the data sheet along with the unrealistic "lab" numbers every networking vendor since the dawn of time has included on their datasheets. Unlike Check Point's competitors, the "real world" performance is evaluated with real-world traffic flows, multiple software blades (security functions), a real-world rulebase, and logging/NAT enabled. Your local Check Point account team can provide more refined appliance recommendations based on your specific requirements.

In Conclusion

As noted before, it's not really clear what use cases Ferro was evaluating ClusterXL against. Even his conclusion is non-specific about this:

Check Point doesn’t have strong solution for clustering and the weak documentation suggests that it’s not fit for critical use cases.

Perhaps Ferro does have a use case that ClusterXL would not be a good fit for. I'd love to know what those "critical use cases" are in more detail so his conclusion could be evaluated more thoroughly. 

Ferro's conclusion is based on an incomplete review of the available information. Only one document was reviewed when others exist, and I question how thoroughly even the one document was reviewed. Only one of the load sharing modes of ClusterXL was reviewed in any detail and he completely ignores the other mode as well as high availability configurations. Ferro also ignores the fact that practically all of Check Point's 100,000+ customers use ClusterXL successfully in their networks, which includes 100% of the Fortune 100 and Global 100 customers across many different industry verticals.

In short: Ferro's review of ClusterXL is incomplete and his conclusion is not supported by the facts.

Disclaimer: I work for Check Point Software Technologies. That said, the above are my own thoughts.

I decided that I'm going to let the phoneboy.net URL expire when it comes up for renewal in a couple of months. As a result of that, the URL for this blog (such that it is) will change to http://securitytheater.phoneboy.com (*Edit*: Now it's http://phoneboy.org, but the securitytheater URLs will redirect) Please update your RSS readers, bookmarks, and the like.

From Stop fretting about mobile security, says Palo Alto Networks founder:

“What I often hear from customers is that 'users have a mobile and they have corporate email and they have Dropbox and I'm afraid they will upload a PDF via Dropbox to their personal account'. Well, what about your Windows users? They've been doing that for the last ten years! Nobody stopped them using Dropbox on their browser for the last ten years.”

So says Nir Zuk, founder and CTO of Palo Alto Networks.

And you know what: he's right. Not necessarily about Dropbox since Dropbox hasn't been around for ten years, but because if you've given people access to a web browser in your organization, you've basically had little to no control over the “applications” they can run. Because even ten years ago, you could run a lot of “applications” organizations so desperately want to control today.

Of course we had URL filtering ten years ago, which can be used to control what people can use with a web browser. But it wasn't as widely used and unless you were using explicit proxies, HTTPS was a pretty big blind spot. And, really, that's only a partial solution since you might want to allow some parts of a web-based application and not others. Doing that solely based on URLs might not always be possible.

But I disagree that you have no control over what end users do on their PCs. Things like the “dead but not going anywhere anytime soon” Anti-Virus/Anti-Malware, firewall, Application Whitelisting, Media Encryption and Port Protection, and a host of other tools, if properly deployed and are monitored, give you something to protect yourself from the malicious things your users get inadvertently from the Internet.

And, of course, segmentation helps too. Not putting your user machines and servers on the same network, using a firewall to media and control access by user, application, service and yes, Nir Zuk, ports.

In fact, once you remember that the browser has made you liable to these kinds of threats for a long time, mobile devices start to look like an attractive option. Zuk claims “mobile devices open up a lot of opportunities for being more secure than today because they do allow the opportunity to control movement of data between devices, and because of the way they're built, the operating system and the controls – especially in iOS 7 and hopefully soon in Android.”

He's absolutely right here. Mobile operating systems are built to be more secure from the ground up. However, you're assuming the device is not rooted or jailbroken, which removes many of the protections these operating systems have in place.

And then there's the data these devices can access and use. What are you doing to ensure data remains protected on these devices? Nir's right there is opportunity to do this better on mobile devices but right now it's an “all or nothing” approach. VDI, Mobile Device Management and secure container technologies are all variations on this approach and users are adverse to all of them.

And then there's the whole lack of visibility over what's going on with the mobile device. At least with a PC you get some, on a mobile device? Not so much.

“You can have a firewall that denies all incoming traffic and bad things still come in,” [Zuk] points out, because Web apps and cloud services mean “the firewall doesn't control access into the network.” Even more bluntly, he's prone to suggesting that “I strongly recommend you take your firewall out and replace it with an Ethernet cable – it will improve the performance and improve the management. And no, I’m not joking.”

Again he's right insofar as replacing a firewall with an Ethernet cable will improve performance and improve management (if you consider removing something to manage an improvement).
However, this advice is utterly clueless as it ignores decades of evidence to the contrary, not to mention the fact Nir Zuk's company Palo Alto Networks sells firewalls.

You know when Windows XP dramatically improved security? In Service Pack 2 when the built-in firewall was enabled by default. Yes, the attacks moved up the stack as a result but a properly configured firewall–even one that only blocks on ports and IPs–is better than no firewall at all.

So should you do about mobile device usage in your enterprise? Depends on your policy and depends on what your critical assets are. Should you “fret” about it? No more than anything else. Just realize mobile devices present unique challenges–and opportunities.

Back when I first got into IT and just started working with FireWall-1, Pointcast was a thing. For those who weren't around back in the mid to late 1990s, Pointcast had a very popular screensaver that displayed news and other information delivered periodically over the Internet to PCs. The problem was: it used an excessive amount of bandwidth on corporate networks, especially if more than a couple of people used it.

The result was, of course, corporations wanted to block access to Pointcast. The problem: how to do it. All we had in the mid 1990s was the traditional firewall which could control access based on IP and port. So we should be able to block the port or IPs it communicates with, right?

Pointcast used good old HTTP. Even back then, no one in their right mind would block HTTP. Of course, everything uses HTTP or HTTPS to communicate these days, and with a traditional firewall with the ability to control traffic only by IP or port, leaving HTTP or HTTPS wide open is tantamount to leaving the barn door open. 

Pointcast didn't exactly publish their list of servers, but users of the PhoneBoy FireWall-1 FAQ contributed a list of IPs plus a couple of other clever solutions to the problem, which I've made available after the break if you're curious.

Of course, with things like content delivery networks, Amazon Web Services, and a host of other ways to serve up an application to users that are available today, attempting to control access to these applications merely by port and IP address is crazy. 

Fortunately, there are a number of solutions to this problem. Check Point's solution is the Application Control Software Blade, which can allow/block access to an application regardless of the ports and destination IP users, and even limit the bandwidth these applications use. New applications or changes to existing applications are made available to the gateway periodically so you can see that you're users are using it and, when it kills you bandwidth or worse, you can block it. 

If only tools like App Control were available back in the day, security admins could have spent more time on more important issues rather than figuring out how to block Pointcast and other applications and I would have a few less FAQ entries on "how do I block X application."

There are a few ways to block access to Pointsec:

  1. Deny HTTP Access to Pointcast Servers
  2. Use the HTTP Security Server
  3. Create a Dummy Host in your DNS/WINS

Deny HTTP Access to Pointcast Servers

To deny HTTP requests to the Pointcast HTTP server, deny access to the following machines:

205.228.184.31 through 205.228.184.60, inclusive. 
206.64.127.31 through 206.64.127.60, inclusive.

To minimize the number of network objects needed (since range objects aren't supported), create the objects as follows and put them into a group:

Create host 205.228.184.31 
Create network 205.228.184.32 with subnet mask 255.255.255.240 (include broadcast) 
Create network 205.228.184.48 with subnet mask 255.255.255.248 (include broadcast) 
Create network 205.228.184.56 with subnet mask 255.255.255.252 (include broadcast) 
Create host 205.228.184.60

Create host 206.64.127.31 
Create network 206.64.127.32 with subnet mask 255.255.255.240 (include broadcast) 
Create network 206.64.127.48 with subnet mask 255.255.255.248 (include broadcast) 
Create network 206.64.127.56 with subnet mask 255.255.255.252 (include broadcast) 
Create host 206.64.127.60

Deny HTTP traffic to these hosts.

Using HTTP Security Server

Thanks to Daniel Blander for this idea:

Create a URI resource that filters the following URLs:

http://*/FIDO*/*

This roughly translates to creating a Wildcard URI Resource with the following parameters:

Service: http 
Action: all 
Host: * 
Path: /FIDO* 
Query: *

You will want to use this URI resource in a rule that denies access.

Create a Dummy Host in your DNS/WINS

Thanks to Mark Syroka for this idea.

Create an entry in your DNS or WINS for the hostname PCNPROXY. Your clients will try and access whatever host resolves to this name if it exists. If you wish to use the PointCast Caching Manager, which is designed to Cache PointCast Requests and is available for free from http://www.pointcast.com/products/intranet/, your DNS/WINS entry would point to this machine. Otherwise, this entry can point to a non-existant machine or any machine that does not run a web server on port 80.