In this article I will take a look at common Check Point specific commands that can help with the administration, troubleshooting and tuning of Check Point firewalls. Note that since only the Gaia platform is actively developped by Check Point, the other legacy platforms such as SPLAT and IPSO will not be handled here.
Check Point has developed it's own 'shell', called clish (or cli.sh). Basically, the clish shell is used for Check Point GAIA device administration, such as configuring host name and interfaces, role based administration, SNMP settings, etc. Basically, the configuration items that can also be configured in the web user interface (WebUI). There is also the possibility to use an actual linux shell, such as bash or sh. In here, there are also Check Point specific commands that can be executed, although those are usually more advanced.
Table of contents:
After a clean install, the default shell is clish. As one can always enter the clish shell by entering 'clish' from the bash shell (or advanced shell, in Check Point terminology), let's configure our admin user to use the 'bash' shell for now. Enter the following commands after logging in to clish:
hostname> set user admin shell /bin/bash hostname> save config hostname> exit
And log in again. You should now be in bash, which can be seen by the [Expert@hostname]# prompt. One tip, if you need your session to stay alive you can disable the auto-timeout in expert mode in the following manner:
[Expert@hostname]# unset TMOUT
Before forcing a failover, it is wise to check the state of the cluster. We expect to have at least one node standby and ready to receive connections when we initiate the failover. Check Point supports stateful failovers, which means that established TCP sessions are kept when the failover occurs (eg: a download over HTTP does not get interrupted). However, before initiating the failover we want to make sure that the number of sessions between the active and standby firewall are more or less equal. Otherwise, the failover might not be as stateful as desired.
One way to force a failover, is to register a critical device as 'down' in the clustering software. The easiest way to do this is to utilize the clusterXL_admin script. Use the following command on an active firewall to force a failover to the standby node:
The problematic device created in this way is temporary, after a reboot the firewall will no longer be marked as 'down'. To make this permanent, use the following:
clusterXL_admin down -p
To see how much entries there are in the connection table, use:
# Get connection table entries fw tab -t connections -s HOST NAME ID #VALS #PEAK #SLINKS localhost connections 8158 23 351 68
In order to find a specific connection, it is best to know the hex value of the IP we're looking for. This conversion can be done with the scripts found over here. Text here
Suppose we want to find the connections involving IP address 10.1.0.200. Converted to hex, this IP is written as 0a0100c8 so this value is used to search the connection table (using grep):
# Get connection table entries for IP 10.1.0.200 fw tab -t connections | grep 0a0100c8 ... <00000000, 0a0100c8, 0000a56e, 0a0100d2, 00000016, 00000006; 0001c001, 00044000, 00000000, 00000e10, 00000000, 5f2fe03c, 00010000, bc934fcf, f6fc93f0, 00000001, 00000001, ffffffff, ffffffff, 00007800, 00000000, 80000084, 00000000, 00000000, 17f26738, ffffc200, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000; 3587/3600> ...
The first field indicates direction, with 0 meaning inbound and 1 outbound. The following four fields are typically the TCP/UDP quadruple of <Source IP, Source Port, Destination IP, Destination Port>. So here, the connection is from source IP 0a0100c8 (10.1.0.200) to 0a0100d2 (10.1.0.210) on service port 0x16 = 22 (SSH). The source port used is 0xa56e = 42350. The last field before the ; indicates the protocol ID (6 for TCP, 17 for UDP). The very last field is the remaining lifetime of the connection, the connection will be removed from the table if the SSH sessions stays idle for another 3587 seconds. More information about the connection table format can be found in sk65133.
This section handles troubleshooting commands, mostly aimed at identifying why the firewalls is dropping certain traffic or is otherwise not functioning as it should.
Sometimes traffic is dropped by the Check Point firewall, but there is no log entry in SmartConsole or the entry doesn't make much sense. Often, more information can be found using a kernel debug. It is recommended to filter the output of the kernel debug command to limit the amount of output, and it is also important to remember that although this debug is generally safe, it is still a kernel debug which can have impact on production traffic. In this example the kernel debug output is filtered to only show drops related to the IP 10.1.0.210:
[Expert@fw1:0]# fw ctl zdebug drop | grep 10.1.0.210 @;5816;[cpu_0];[fw4_1];fw_log_drop_ex: Packet proto=6 184.108.40.206:50588 -> 10.1.0.210:80 dropped by fw_send_log_drop Reason: Rulebase drop - DEFAULT POLICY; @;5848;[cpu_0];[fw4_0];fw_log_drop_ex: Packet proto=6 220.127.116.11:50589 -> 10.1.0.210:8000 dropped by fw_send_log_drop Reason: Rulebase drop - DEFAULT POLICY;
Sometimes with Check Point you can run into unexplainable NAT issues. Following the tip above, a kernel debug (fw ctl zdebug drop) is run to investigate the issue. One might encounter the following error messages:
fw_log_drop_ex: Packet proto=17 18.104.22.168:67 -> 22.214.171.124:67 dropped by fw_conn_inspect Reason: post lookup verification failed;
In this case it might be a case of corruption of the connection table. The connection table can be cleared using the following commands:
[Expert@vsx-gw02:1]# fw tab -t connections -x This will clear all the entries in table connections !!! Are you sure (yes/no)? [n] yes Clearing table connections [Expert@vsx-gw-02:1]#