NG Firewall Filter Apps
Ad Blocker
Ad Blocker allows you to block a majority of advertising content that is delivered to users on web pages that they request.
![](/assets/images/user-manual/um-etm-me/Graphics/Ad%20Blocker/AdBlocker.png)
Ad Blocker uses downloadable filter subscriptions from a variety of sources which contain lists of web sites and extensions that are typically used to deliver advertising.
Settings
This section reviews the different settings and configuration options available for Ad Blocker.
Status
![](/assets/images/user-manual/um-etm-me/Graphics/Ad%20Blocker/apps_ad-blocker_status.png)
Options
If Block Ads is enabled, then Ad Blocker will block web requests that it determines are for advertisements. It will return an HTTP 403 permission denied to the client that requests the ad.
If Block Tracking & Ad Cookies is enabled Ad Blocker will block cookies that are used for advertising or behavior tracking purposes.
The Update button will update the ad filters signatures.
![](/assets/images/user-manual/um-etm-me/Graphics/Ad%20Blocker/apps_ad-blocker_options.png)
Ad Filters
![](/assets/images/user-manual/um-etm-me/Graphics/Ad%20Blocker/apps_ad-blocker_ad-filters.png)
Cookie Filters
![](/assets/images/user-manual/um-etm-me/Graphics/Ad%20Blocker/apps_ad-blocker_cookie-filters.png)
Pass Lists
- Passed Sites: Any domains you add to the Passed Sites list will allow ads and cookies, even if blocked by an existing filter - just add the domain and save. Unchecking the pass option will allow ads to be blocked as if the entry was not present.
- Passed Client IPs: If you add an IP to this list, Ad Blocker will not block any ads or cookies from that IP. Just add the IP and save. Unchecking the pass option will have the block/pass lists affect the user as if they were not entered into the Passed Client IPs list.
![](/assets/images/user-manual/um-etm-me/Graphics/Ad%20Blocker/apps_ad-blocker_pass-lists.png)
Reports
The Reports tab provides a view of all reports and events for all traffic handled by Ad Blocker.
This applications reports can be accessed via the Reports tab at the top or the Reports tab within the settings. All pre-defined reports will be listed along with any custom reports that have been created.
Reports can be searched and further defined using the time selectors and the Conditions window at the bottom of the page. The data used in the report can be obtained on the Current Data window on the right.
Pre-defined report queries: report Entry | Description |
---|---|
Ad Blocker Summary | A summary of ad blocker actions. |
Ads Blocked | The amount of detected and blocked ads over time. |
Top Blocked Ad Sites | The number of blocked ads grouped by website. |
All Ad Events | All HTTP requests are scanned by Ad Blocker. |
Blocked Ad Events | HTTP requests blocked by Ad Blocker. |
Blocked Cookie Events | Requests blocked by cookie filters. |
Related Topics
Ad Blocker Reports
The Reports tab provides a view of all reports and events for all traffic handled by Ad Blocker.
Reports
This application's reports can be accessed via the Reports tab at the top or the Reports tab within the settings. All pre-defined reports will be listed along with any custom reports that have been created.
Reports can be searched and further defined using the time selectors and the Conditions window at the bottom of the page. The data used in the report can be obtained on the Current Data window on the right.
Report Entry | Description |
---|---|
Ad Blocker Summary | A summary of ad blocker actions. |
Ads Blocked | The amount of detected and blocked ads over time. |
Top Blocked Ad Sites | The number of blocked ads grouped by website. |
All Ad Events | All HTTP requests are scanned by Ad Blocker. |
Blocked Ad Events | HTTP requests blocked by Ad Blocker. |
Blocked Cookie Events | Requests blocked by cookie filters. |
Related Topics
Application Control
Application Control leverages the Network Application Visibility Library (NAVL) from Procera Networks [1] to perform deep packet (DPI) and deep flow (DFI) inspection of network traffic. This allows the server to accurately identify thousands of today's common applications such as Social Networking, P2P, Instant Messaging, Video Streaming, File Sharing, Enterprise Applications, Web 2.0, and much more.
![](/assets/images/user-manual/um-etm-me/Graphics/Application%20Control/ApplicationControl.png)
About Application Control
For most common applications, you can simply go to the list on the Applications tab and check Block for anything you want to stop. Then, Application Control will take care of the rest. If you need a higher degree of control, you can use the Rules tab to create custom rules that target more complex traffic patterns.
How It Works
Application Control feeds each chunk of data to a classification engine as it passes through the application. The classification engine continues to analyze the traffic flow and keeps properties of the session, such as the Application property. Each time the classification of the Application property is updated, the Applications settings are checked to see if that application is allowed. The data is blocked If the application is configured to be blocked in the settings. If not, the process continues until the session reaches a fully classified state, where the classification engine believes no more classification of the session is possible. At this point, the Rules are evaluated, and the session is ultimately blocked or passed based on the rules you've configured.
Settings
This section describes the different settings and configuration options available for Application Control.
Status
![](/assets/images/user-manual/um-etm-me/Graphics/Application%20Control/apps_application-control_status.png)
Applications
- Application: The unique identifier for the application.
- Block: Enable this checkbox to block/reset sessions of this application. For TCP, this will actively reset the connection. For UDP, the packet will be dropped, and the session will be killed.
- Tarpit: Enable this checkbox to block/tarpit sessions of this application. For TCP, this makes it appear to both the client and the server that the other party is receiving the data, but it is not responsive. It silently drops the data. For UDP, it is identical in behavior to block, except the connection is kept open, so that the next packet will be dropped instead of recategorized as a new session.
- Flag: Enable the checkbox to flag the traffic. It will be flagged as a violation in Reports.
- Name: The standard name for the application.
- Category: A fairly general and high level category for the application.
- Productivity: Productivity is best thought of as an index value between 1 and 5 that rates the potential for each application to improve or increase the overall productivity of your network users, assuming, of course, that listening to music and playing online games is not in their job description. So, applications with a low Productivity index (e.g. MySpace, Hulu, Zynga Games) can be expected to have a negative impact on productivity. Items with a high value (e.g. Active Directory, Network File System) can generally be considered critical for maintaining or improving productivity.
- Risk: Risk is another index value between 1 and 5 that rates the potential for each protocol or application to allow nasty stuff onto your network. The higher the risk index, the greater the chance of letting in something that could be dangerous or destructive. So low risk items (e.g. Active Directory, Oracle, LDAP) are generally no cause for concern, while applications rated with a high risk (e.g. BitTorrent, Pando, Usenet) increase the possibility you'll find yourself spending long nights deleting pirated software and cleaning up viruses and other exploits that find their way into your infrastructure.
- Description: Provides a more detailed description of each application on the list. In some cases the description is much larger than will fit within the grid column, so you can click on any description to see a pop-up window with the full text displayed.
![](/assets/images/user-manual/um-etm-me/Graphics/Application%20Control/apps_application-control_applications.png)
Rules
If the traffic you need to manage can't be handled via the Applications tab, you can create custom rules that will allow you to analyze and control traffic based on much more complex patterns and conditions. For each session, the rules are only evaluated once after the classification engine has completed an analysis of the traffic. The rules are then evaluated in order until the first match is found; at this point, the configured action will be performed. If there are no matches, the session will be tagged as allowed, the traffic will flow unimpeded, and no further analysis of that traffic will occur.
If an application is blocked or tarpitted in the Applications tab, it will be blocked immediately when identified before the engine has completed the analysis. In this case, the rules will have NO EFFECT because the sessions are blocked before the rules are evaluated.
![](/assets/images/user-manual/um-etm-me/Graphics/Application%20Control/apps_application-control_rules.png)
![](/assets/images/user-manual/um-etm-me/Graphics/Application%20Control/apps_application-control_applications.png)
Anatomy of a Rule
- The first matcher makes sure the rule only looks at TCP traffic.
- The second causes the rule to only look at traffic with a destination port 443.
- The third matcher is where the real magic starts. In this case, we created a Glob matcher that looks for the /SSL tag anywhere in the Application Control/ProtoChain. (Don't worry, we'll cover globs and chains below!)
- The fourth matcher is the frosting on the cake. We tell the rule to look at the Application Control/Detail parameter. This is where the server name from the SSL certificate will be located when an SSL encrypted session is detected. In this case, we left the Value field empty since we're looking for cases where there is no valid certificate.
Application Detail
Matcher | Detail Contents | Example |
---|---|---|
Application: FBOOKAPP | The name of the Facebook Application that is being accessed. | wordswithfriends |
Application: HTTP | The contents of the Content-Type header in the session data coming from the server. | image/jpg |
ProtoChain: */SSL* | The server name extracted from the SSL certificate used to encrypt the session. | www.gmail.com |
- Allow: Allow the traffic.
- Block: When this option is selected, traffic in both directions will be silently dropped, but the session will remain active.
Reports
The Reports tab provides a view of all reports and events for all traffic handled by Application Control.
This applications reports can be accessed via the Reports tab at the top or the Reports tab within the settings. All pre-defined reports will be listed along with any custom reports that have been created.
Reports can be searched and further defined using the time selectors and the Conditions window at the bottom of the page. The data used in the report can be obtained on the Current Data window on the right.
Report Entry | Description |
---|---|
Application Control Summary | A summary of Application Control actions. |
Top Applications Usage | The amount of bandwidth per top application. |
Scanned Sessions (all) | The amount of scanned, flagged, and blocked sessions over time. |
Scanned Sessions (flagged) | The amount of flagged, and blocked sessions over time. |
Scanned Sessions (blocked) | The amount of flagged, and blocked sessions over time. |
Top Categories (by sessions) | The number of sessions grouped by category. |
Top Applications (by sessions) | The number of sessions grouped by application. |
Top Applications (by size) | The number of bytes grouped by application. |
Top Flagged Applications | The number of flagged sessions grouped by application. |
Top Blocked Applications | The number of blocked sessions grouped by application. |
Top Flagged Hostnames | The number of flagged sessions grouped by hostname. |
Top Blocked Hostnames | The number of blocked sessions grouped by hostname. |
Top Flagged Clients | The number of flagged sessions grouped by client. |
Top Blocked Clients | The number of blocked sessions grouped by client. |
Top Flagged Usernames | The number of flagged sessions grouped by username. |
Top Blocked Usernames | The number of blocked sessions grouped by username. |
Classified Sessions | All sessions matching an application control signature. |
Flagged Sessions | All sessions matching an application control signature and flagged. |
Blocked Sessions | All sessions matching an application control signature and blocked. |
All Sessions | All sessions are scanned by Application Control. |
Related Topics
Application Control Reports
The Reports tab provides a view of all reports and events for all traffic handled by Application Control.
Reports
This applications reports can be accessed via the Reports tab at the top or the Reports tab within the settings. All pre-defined reports will be listed along with any custom reports that have been created.
Reports can be searched and further defined using the time selectors and the Conditions window at the bottom of the page. The data used in the report can be obtained on the Current Data window on the right.
Report Entry | Description |
---|---|
Application Control Summary | A summary of Application Control actions. |
Top Applications Usage | The amount of bandwidth per top application. |
Scanned Sessions (all) | The amount of scanned, flagged, and blocked sessions over time. |
Scanned Sessions (flagged) | The amount of flagged, and blocked sessions over time. |
Scanned Sessions (blocked) | The amount of flagged, and blocked sessions over time. |
Top Categories (by sessions) | The number of sessions grouped by category. |
Top Applications (by sessions) | The number of sessions grouped by application. |
Top Applications (by size) | The number of bytes grouped by application. |
Top Flagged Applications | The number of flagged sessions grouped by application. |
Top Blocked Applications | The number of blocked sessions grouped by application. |
Top Flagged Hostnames | The number of flagged sessions grouped by hostname. |
Top Blocked Hostnames | The number of blocked sessions grouped by hostname. |
Top Flagged Clients | The number of flagged sessions grouped by client. |
Top Blocked Clients | The number of blocked sessions grouped by client. |
Top Flagged Usernames | The number of flagged sessions grouped by username. |
Top Blocked Usernames | The number of blocked sessions grouped by username. |
Classified Sessions | All sessions matching an application control signature. |
Flagged Sessions | All sessions matching an application control signature and flagged. |
Blocked Sessions | All sessions matching an application control signature and blocked. |
All Sessions | All sessions scanned by Application Control. |
Application Control Lite
Application Control Lite scans sessions for the specified signatures and will log and/or block sessions based on their content. Many applications can be hard to block based on port, as modern applications will find and use open ports automatically. Application Control Lite provides a basic functionality to block sessions based on the content (data) in the session.
Settings
This section reviews the different settings and configuration options available for Application Control Lite.
Status
![](/assets/images/user-manual/um-etm-me/Graphics/Application%20Control%20Lite/apps_application-control-lite_status.png)
Signatures
The Signatures tab shows the list of current signatures. Signatures are regular expressions written to match known protocols as accurately as possible. New custom signatures can be designed to match certain applications or sessions, or signatures can often be found for many existing protocols L7-filter Supported Protocols.
As the early data in each session goes from the server to the client and the client to the server, it is stored in a buffer. As each chunk of data arrives, the data is evaluated against any enabled signatures. If the signature is checked "log" then the session will be tagged and logged as having matched the specified signature. If the signature is checked "block" it will be logged, and the session will immediately be closed.
- It will not match anything. In this case, the signature needs to be fixed.
- It will block the desired protocol/application and nothing else. This is ideal.
- It will only partially block the protocol. Many multi-session protocols only have some sessions identified. This can have varying effects depending on the application.
- It will block the protocol and block other things too (false positives). This can cause major problems with the network.
- It will block the protocol, and the application will adapt and use an alternative protocol to communicate. Many applications will try alternative techniques to avoid blocking.
Reporting
The Reports tab provides a view of all reports and events for all traffic handled by Application Control Lite.
Reports
This applications reports can be accessed via the Reports tab at the top or the Reports tab within the settings. All pre-defined reports will be listed along with any custom reports that have been created.
Reports can be searched and further defined using the time selectors and the Conditions window at the bottom of the page. The data used in the report can be obtained on the Current Data window on the right.
Report Entry | Description |
---|---|
Application Control Lite Summary | A summary of Application Control Lite actions. |
Detection Statistics | The number of logged and blocked sessions over time. |
Top Blocked Protocols | The top blocked sessions by protocol. |
Top Logged Protocols | The top logged sessions by protocol. |
Top Blocked Hosts | The top blocked sessions by host. |
Top Logged Hosts | The top logged sessions by host. |
Top Blocked Users | The top blocked sessions by user. |
Top Logged Users | The top logged sessions by user. |
All Events | All sessions scanned by Application Control Lite. |
Blocked Events | All sessions matching an application signature and blocked. |
Related Topics
Application Control Lite Reports
The Reports tab provides a view of all reports and events for all traffic handled by Application Control Lite.
Reports
This applications reports can be accessed via the Reports tab at the top or the Reports tab within the settings. All pre-defined reports will be listed along with any custom reports that have been created.
Reports can be searched and further defined using the time selectors and the Conditions window at the bottom of the page. The data used in the report can be obtained on the Current Data window on the right.
Report Entry | Description |
---|---|
Application Control Lite Summary | A summary of Application Control Lite actions. |
Detection Statistics | The number of logged and blocked sessions over time. |
Top Blocked Protocols | The top blocked sessions by protocol. |
Top Logged Protocols | The top logged sessions by protocol. |
Top Blocked Hosts | The top blocked sessions by host. |
Top Logged Hosts | The top logged sessions by host. |
Top Blocked Users | The top blocked sessions by user. |
Top Logged Users | The top logged sessions by user. |
All Events | All sessions are scanned by Application Control Lite. |
Blocked Events | All sessions matching an application signature and blocked. |
SSL Inspector
The SSL Inspector is a special application that allows other NG Firewall applications that process HTTP traffic to also process encrypted HTTPS traffic and applications that process SMTP to also process SMTP over SSL. It does this by performing man-in-the-middle decryption and encryption of SSL traffic, passing the unencrypted traffic through the NG Firewall server for inspection by other applications and services.
![](/assets/images/user-manual/um-etm-me/Graphics/SSL%20Inspector/SSLInspector.png)
About SSL Inspector
When a client makes an HTTPS request, the Inspector first initiates a secure SSL connection with the external server on behalf of the client. While this session is being established, the inspector captures information about the server SSL certificate. Once the server session is active, the Inspector uses the details from the server certificate to create a new certificate that will be used to encrypt the session between the inspector and the client. This certificate is generated or loaded on the fly, and is created using the same subject details contained in the actual server certificate. The certificate is then signed by the internal CA on the NG Firewall server and is used to establish a secure connection between the inspector and the client. Creating the certificate this way is necessary to eliminate security warnings on the client, but it does require a few extra steps to properly configure the client computers and devices on your network. See the SSL Certificates section below for details.
SSL Certificates
SSL Certificates serve two primary purposes. They allow traffic between the client and server to be encrypted, and they allow the client to validate the authenticity of the server. There are two main ways the client checks the authenticity of the server certificate. The first is by validating the server certificate to ensure it has been issued or signed by a known and trusted third party certificate authority. Once that trust has been established, the client checks the server name portion of the target URL to ensure it matches the server name registered in the certificate presented by the server. If either of these checks fail, the client will typically display a warning, indicating that the security of the connection may be compromised.
When the NG Firewall server is initially installed, a default Certificate Authority is created automatically and used to sign the man-in-the-middle certificates created by the SSL Inspector. To view or make changes to the internal Certificate Authority, check out Certificates tab of the Config/Administration page.
Client Configuration
For the client authenticity checks to be successful, the client must be configured to trust the root certificate used by the NG Firewall server to sign the man-in-the-middle certificates described above. To configure clients, you must first use the Download Root Certificate button located on the Configuration tab of SSL Inspector Settings page to download the root certificate. You must then install this certificate to correct the location on the client.
Another way to download the root certificate is to simply access a special URL using the IP address of the NG Firewall server:
http://yourserver/cert
Simply replace 0.0.0.0 with the IP address of your NG Firewall server. This method is especially useful when using mobile devices. For example, accessing this URL on an iPad or iPhone will download and display the certificate, and provide an option to install and trust the certificate directly on the device.
Below are basic instructions for installing the root certificate on some common client platforms. If yours is not listed, or you have any difficulty, consult the reference material for the target platform for further information.
- Log into the NG Firewall server running SSL Inspector.
- Go to Download Root Certificate Authority (CA)" button. and download the certificate using the "
- Copy the root_authority.crt you just downloaded to the Windows client computer.
- From a command prompt, or from Start/Run, run the command "certmgr.msc".
- Open the "Trusted Root Certification Authorities" tree in the panel on the left.
- Right click on "Certificates" and select .
- Proceed with the Certificate Import Wizard, selecting the the root_authority.crt file.
- Log into the NG Firewall server running SSL Inspector.
- Go to Download Root Certificate Authority (CA)" button. and download the certificate using the "
- Copy the root_authority.crt you just downloaded to the Windows client computer.
- Launch Firefox
- From the Tools menu, go to .
- Click the Import button and select the root_authority.crt file.
- Enable the "Trust this CA to identify websites" checkbox and click the OK button.
- Log into the NG Firewall server running SSL Inspector.
- Go to Download Root Certificate Authority (CA) button. and download the certificate using the
- Copy the root_authority.crt you just downloaded to the Windows client computer.
- Launch Opera
- From the Tools menu, go to Manage Certificates. and click
- Select the Authorities tab, click Import, and select the root_authority.crt file.
- Click Install and click OK when asked if you want to trust the certificate.
Group Policy Distribution
If you have a fully deployed and implemented Active Directory infrastructure, you can leverage the Group Policy model to distribute the NG Firewall root certificate to all of your client computers. This is way outside our area of expertise, so we can't provide much help or assistance, but we have compiled links to some TechNet articles with instructions for several common versions of Windows Server.
Settings
This section describes the different settings and configuration options available for SSL Inspector.
Status
![](/assets/images/user-manual/um-etm-me/Graphics/SSL%20Inspector/apps_ssl-inspector_status.png)
Configuration
Execute the following configurations:
Download Root Certificate
As described above, client computers and devices on your network need to be configured to trust the root certificate of the NG Firewall server. Clicking this button will allow you to download the root certificate. Once downloaded, you need to install it in the Trusted Authorities certificate store on your client computers and devices. Note that this is the same root certificate that can be downloaded from the
page. The download link is included on the SSL Inspector Configuration page for convenience.Alternatively, you can download the certificate from a client system by navigating to http://yourserver/cert.
Enable SMTPS Traffic Processing
This option is enabled by default and allows the SSL Inspector to work cooperatively with the other applications that act on SMTP mail traffic. When enabled, port 25 mail sessions that use STARTTLS will be decrypted inbound, allowing the clear traffic to pass through all other applications, and the re-encrypted again before passing outbound.
Enable HTTPS Traffic Processing
This option is enabled by default, and allows the SSL Inspector to work cooperatively with the other applications that act on HTTP web traffic. When enabled, port 443 web sessions that use SSL/TLS will be decrypted inbound, allowing the clear traffic to pass through all other applications, and then re-encrypted again before passing outbound.
Block Invalid HTTPS Traffic
When processing a new HTTPS session, the first thing the inspector does is analyze the initial client request to see if it contains a valid SSL negotiation message. If not, by default the session will be ignored, and the traffic will flow directly between the client and server with no inspection performed. By enabling this checkbox, you can change the default behavior and effectively block any port 443 traffic that does not contain a valid HTTPS signature.
Client/Server Connection Protocols
- SSLv2Hello - This is really a legacy handshake protocol that is used between a client and server when deciding which encryption protocol to use. This means it's possible to enable SSLv2Hello and still have a TLSv1.x connection negotiated. While there are no known security issues, we still recommend leaving this disabled unless you specifically need this legacy support.
- SSLv3 - This is an older protocol that is been deprecated since the discovery of the POODLE vulnerability. For that reason, we recommend this be disabled for maximum security.
- TLSv1 - This is an older protocol that has some known weaknesses. These can be mitigated if the other side of the connection forces certain secure ciphers to be used. However, since this can't be guaranteed, best practice is to disable this protocol unless it is required to support connections with legacy clients or servers.
- TLSv1.1 - This is a modern protocol that is generally regarded as secure and is used as a fallback for 1.2 or in older browsers.
- TLSv1.2 - This is the most common and recommended TLS version.
- TLSv1.3- This is the most recent version of the TLS protocol and offers the highest security but may have issues with some websites.
Trust All Server Certificates
Normally, when establishing an SSL connection with an external web server, the inspector will authenticate the server certificate against a standard list of trusted certificate authorities. If this trust cannot be established, the inspector will end the session. By enabling this checkbox, you can force the inspector to blindly trust all external server certificates.
Please note that we DO NOT recommend running with this option enabled, as it exposes all HTTPS traffic to significant security risks.
The standard list of trusted certificates used by NG Firewall is generated from the standard ca-certificates package. It includes, among others, certificate authorities used by Mozilla's browsers. Please note that Edge Threat Management staff can neither confirm nor deny whether the certificate authorities whose certificates are included in this list have in any way been audited for trustworthiness or RFC 3647 compliance. Full responsibility to assess them belongs to the local system administrator.
Upload Trusted Certificate
![](/assets/images/user-manual/um-etm-me/Graphics/SSL%20Inspector/apps_ssl-inspector_configuration.png)
Rules
The Rules tab allows you to specify explicit rules to Inspect or Ignore HTTPS traffic that crosses the NG Firewall. By default, many common HTTPS sites (Google, YouTube, Yahoo, and so on) are inspected, but not all HTTPS. This provides a safe default which provides HTTPS inspection on those sites without interfering with other HTTPS communications. It can easily be configured to inspect all HTTPS by enabling the "Inspect All Traffic" rule.
The Rules describes how rules work and how they are configured. SSL Inspector uses rules to determine if it should inspect or ignore traffic for the specific session.
In addition to all the common rule types, there are three that are unique to the SSL Inspector, and these can be very useful for ignoring traffic that you don't want to inspect or that isn't compatible with the SSL Inspector.
HTTPS: SNI Hostname
Most web browsers and many client applications include the destination hostname in the initial packet of an HTTPS session. The mechanism is called the Server Name Indication or the SNI extension to the TLS protocol. The main purpose is to allow a single web server to host multiple secure web sites. By analyzing the SNI hostname in the client request, the server can decide which SSL certificate to use for encrypting the session. This extension is necessary because the encryption must be established long before the server ever sees the HTTP request, and by then, it would be too late to use a different certificate.
Creating ignore rules based on the SNI hostname is an effective way to have the SSL Inspector ignore incompatible traffic. A prime example is the default rule for Microsoft Update. The Microsoft Update client checks the server certificate to ensure it was signed by a specific authority. Since it doesn't trust the Root Authority the SSL Inspector uses to generate certificates on-the-fly, Microsoft Update will fail with an error. The default rule allows this traffic to be detected and ignored, allowing Microsoft Update to work properly.
HTTPS: Certificate Subject and HTTPS: Certificate Issuer
These two rule conditions are useful when dealing with client applications that don't use SNI and aren't compatible with SSL Inspector. An excellent example is the Dropbox client utility, for which there is also a default rule. Like Microsoft Update, the Dropbox client will reject SSL certificates it doesn't explicitly trust.
Using either of these rule conditions, you can match traffic on any portion of the Subject or Issuer Distinguished Name (DN) included in the server certificate. In both cases, the information in the match string includes the standard information fields commonly stored within the SSL certificates, such as CN (common name), C (country), ST (state), L (locality), O (organization), and OU (organizational unit). Each of these is appended to the match string and separated by commas. Note that not all fields are required in all certificates, and some certificates may have others not listed. The order they occur in the match string is also not guaranteed.
CN=*.dropbox.com, O="Dropbox, Inc.", L=San Francisco, ST=California, C=US
CN=Thawte SSL CA, O="Thawte, Inc.", C=US
- Inspect: Causes the traffic which matched the rule to be decrypted and passed along to other applications and service for further inspection, classification, and possible action.
- Ignore: Causes the traffic which matched the rule to be ignored by the SSL Inspector.
Figure 6. SSL Inspector Rules
Reports
The Reports tab provides a view of all reports and events for all traffic handled by HTTPS Inspector.
This applications reports can be accessed via the Reports tab at the top or the Reports tab within the settings. All pre-defined reports will be listed along with any custom reports that have been created.
Reports can be searched and further defined using the time selectors and the Conditions window at the bottom of the page. The data used in the report can be obtained on the Current Data window on the right.
Report Entry | Description |
---|---|
SSL Inspector Summary | A summary of SSL Inspector actions. |
Sessions Scanned | The amount of SSL sessions over time. |
Sessions Inspected | The amount of inspected SSL sessions over time. |
Top Inspected Sites | The number of inspected sessions grouped by site. |
Top Ignored Sites | The number of ignored sessions grouped by site. |
All Sessions | All sessions detected by SSL Inspector. |
Inspected Sessions | Events where traffic was fully processed by the inspector, and all traffic was passed through all the other applications and services. |
Ignored Sessions | Events where traffic was not or could not be inspected, so the traffic was completely ignored and not analyzed by any applications or services. |
Blocked Sessions | Events where traffic was blocked because it did not contain a valid SSL request and the Block Invalid Traffic option was enabled. |
Untrusted Sessions | Events where traffic was blocked because the server certificate could not be authenticated. |
Abandoned Sessions | Events where traffic was blocked due to an underlying problems with the SSL session. |
The tables queried to render these reports:
Status
- INSPECTED means the session was fully processed by the inspector, and all traffic was passed through all the other applications and services.
- IGNORED means the session was not or could not be inspected, so the traffic was completely ignored and not analyzed by any applications or services.
- BLOCKED means the traffic was blocked because it did not contain a valid HTTPS request, and the Block Invalid Traffic option was enabled.
- UNTRUSTED means the traffic was blocked because the server certificate could not be authenticated.
- ABANDONED means the connection failed because an an underlying SSL connection problem. Usually that the client abandoned the connection because the certificate was not trusted.
Detail
Extra details about the session, with the exact content dependent on the event status.
For INSPECTED and UNTRUSTED sessions, this field will include the SNI hostname extracted from the initial message sent from the client to the server. If the SNI information is not available, the server IP address will be used instead.
For BLOCKED or IGNORED sessions, this field will contain the description of the rule that matched and was applied to the session.
SSL Exception Message | Description |
---|---|
unexpected_message | An inappropriate message was received. This alert is always fatal and should never be observed in communication between proper implementations. |
bad_record_mac | This alert is returned if a record is received with an incorrect MAC. This alert also MUST be returned if an alert is sent because a TLSCiphertext decrypted in an invalid way: either it wasn't an even multiple of the block length, or its padding values, when checked, weren't correct. This message is always fatal and should never be observed in communication between proper implementations (except when messages were corrupted in the network). |
decryption_failed | This alert was used in some earlier versions of TLS, and may have permitted certain attacks against the CBC mode [CBCATT]. It MUST NOT be sent by compliant implementations. |
record_overflow | A TLSCiphertext record was received that had a length more than 2^14+2048 bytes, or a record decrypted to a TLSCompressed record with more than 2^14+1024 bytes. This message is always fatal and should never be observed in communication between proper implementations (except when messages were corrupted in the network). |
decompression_failure | The decompression function received improper input (e.g., data that would expand to excessive length). This message is always fatal and should never be observed in communication between proper implementations. |
handshake_failure | Reception of a handshake_failure alert message indicates that the sender was unable to negotiate an acceptable set of security parameters given the options available. This is a fatal error. |
no_certificate | This alert was used in SSLv3 but not any version of TLS. It MUST NOT be sent by compliant implementations. |
bad_certificate | A certificate was corrupt, contained signatures that did not verify correctly, etc. |
unsupported_certificate | A certificate was of an unsupported type. |
certificate_revoked | A certificate was revoked by its signer. |
certificate_expired | A certificate has expired or is not currently valid. |
certificate_unknown | Some other (unspecified) issue arose in processing the certificate, rendering it unacceptable. |
illegal_parameter | A field in the handshake was out of range or inconsistent with other fields. This message is always fatal. |
unknown_ca | A valid certificate chain or partial chain was received, but the certificate was not accepted because the CA certificate could not be located or couldn't be matched with a known, trusted CA. This message is always fatal. |
access_denied | A valid certificate was received, but when access control was applied, the sender decided not to proceed with negotiation. This message is always fatal. |
decode_error | A message could not be decoded because some field was out of the specified range or the length of the message was incorrect. This message is always fatal and should never be observed in communication between proper implementations (except when messages were corrupted in the network). |
decrypt_error | A handshake cryptographic operation failed, including being unable to correctly verify a signature or validate a Finished message. This message is always fatal. |
export_restriction | This alert was used in some earlier versions of TLS. It MUST NOT be sent by compliant implementations. |
protocol_version | The protocol version the client has attempted to negotiate is recognized but not supported. (For example, old protocol versions might be avoided for security reasons.) This message is always fatal. |
insufficient_security | Returned instead of handshake_failure when a negotiation has failed specifically because the server requires ciphers more secure than those supported by the client. This message is always fatal. |
internal_error | An internal error unrelated to the peer or the correctness of the protocol (such as a memory allocation failure) makes it impossible to continue. This message is always fatal. |
user_canceled | This handshake is being canceled for some reason unrelated to a protocol failure. If the user cancels an operation after the handshake is complete, just closing the connection by sending a close_notify is more appropriate. This alert should be followed by a close_notify. This message is generally a warning. |
no_renegotiation | Sent by the client in response to a hello request or by the server in response to a client hello after initial handshaking. Either of these would normally lead to renegotiation; when that is not appropriate, the recipient should respond with this alert. At that point, the original requester can decide whether to proceed with the connection. One case where this would be appropriate is where a server has spawned a process to satisfy a request; the process might receive security parameters (key length, authentication, etc.) at startup, and it might be difficult to communicate changes to these parameters after that point. This message is always a warning. |
unsupported_extension | sent by clients that receive an extended server hello containing an extension that they did not put in the corresponding client hello. This message is always fatal. |
SSL Inspector Reports
The Reports tab provides a view of all reports and events for all traffic handled by HTTPS Inspector.
Reports
This applications reports can be accessed via the Reports tab at the top or the Reports tab within the settings. All pre-defined reports will be listed along with any custom reports that have been created.
Reports can be searched and further defined using the time selectors and the Conditions window at the bottom of the page. The data used in the report can be obtained on the Current Data window on the right.
Pre-defined report queries:
Report Entry | Description |
---|---|
SSL Inspector Summary | A summary of SSL Inspector actions. |
Sessions Scanned | The amount of SSL sessions over time. |
Sessions Inspected | The amount of inspected SSL sessions over time. |
Top Inspected Sites | The number of inspected sessions grouped by site. |
Top Ignored Sites | The number of ignored sessions grouped by site. |
All Sessions | All sessions detected by SSL Inspector. |
Inspected Sessions | Events where traffic was fully processed by the inspector, and all traffic was passed through all the other applications and services. |
Ignored Sessions | Events where traffic was not or could not be inspected, so the traffic was completely ignored and not analyzed by any applications or services. |
Blocked Sessions | Events where traffic was blocked because it did not contain a valid SSL request and the Block Invalid Traffic option was enabled. |
Untrusted Sessions | Events where traffic was blocked because the server certificate could not be authenticated. |
Abandoned Sessions | Events where traffic was blocked due to an underlying problems with the SSL session. |
The tables queried to render these reports:
Status
- INSPECTED means the session was fully processed by the inspector, and all traffic was passed through all the other applications and services.
- IGNORED means the session was not or could not be inspected, so the traffic was completely ignored and not analyzed by any applications or services.
- BLOCKED means the traffic was blocked because it did not contain a valid HTTPS request, and the Block Invalid Traffic option was enabled.
- UNTRUSTED means the traffic was blocked because the server certificate could not be authenticated.
- ABANDONED means the connection failed because an an underlying SSL connection problem. Usually, the client abandoned the connection because the certificate was not trusted.
Detail
Extra details about the session, with the exact content dependent on the event status.
For INSPECTED and UNTRUSTED sessions, this field will include the SNI hostname extracted from the initial message sent from the client to the server. If the SNI information is not available, the server IP address will be used instead.
For BLOCKED or IGNORED sessions, this field will contain the description of the rule that matched and was applied to the session.
SSL Exception Messages | Description |
---|---|
unexpected_message | An inappropriate message was received. This alert is always fatal and should never be observed in communication between proper implementations. |
bad_record_mac | This alert is returned if a record is received with an incorrect MAC. This alert also MUST be returned if an alert is sent because a TLSCiphertext decrypted in an invalid way: either it wasn't an even multiple of the block length, or its padding values, when checked, weren't correct. This message is always fatal and should never be observed in communication between proper implementations (except when messages were corrupted in the network). |
decryption_failed | This alert was used in some earlier versions of TLS, and may have permitted certain attacks against the CBC mode [CBCATT]. It MUST NOT be sent by compliant implementations. |
record_overflow | A TLSCiphertext record was received that had a length more than 2^14+2048 bytes, or a record decrypted to a TLSCompressed record with more than 2^14+1024 bytes. This message is always fatal and should never be observed in communication between proper implementations (except when messages were corrupted in the network). |
decompression_failure | The decompression function received improper input (e.g., data that would expand to excessive length). This message is always fatal and should never be observed in communication between proper implementations. |
handshake_failure | Reception of a handshake_failure alert message indicates that the sender was unable to negotiate an acceptable set of security parameters given the options available. This is a fatal error. |
no_certificate | This alert was used in SSLv3 but not any version of TLS. It MUST NOT be sent by compliant implementations. |
bad_certificate | A certificate was corrupt, contained signatures that did not verify correctly, etc. |
unsupported_certificate | A certificate was of an unsupported type. |
certificate_revoked | A certificate was revoked by its signer. |
certificate_expired | A certificate has expired or is not currently valid. |
certificate_unknown | Some other (unspecified) issue arose in processing the certificate, rendering it unacceptable. |
illegal_parameter | A field in the handshake was out of range or inconsistent with other fields. This message is always fatal. |
unknown_ca | A valid certificate chain or partial chain was received, but the certificate was not accepted because the CA certificate could not be located or couldn't be matched with a known, trusted CA. This message is always fatal. |
access_denied | A valid certificate was received, but when access control was applied, the sender decided not to proceed with negotiation. This message is always fatal. |
decode_error | A message could not be decoded because some field was out of the specified range or the length of the message was incorrect. This message is always fatal and should never be observed in communication between proper implementations (except when messages were corrupted in the network). |
decrypt_error | A handshake cryptographic operation failed, including being unable to correctly verify a signature or validate a Finished message. This message is always fatal. |
export_restriction | This alert was used in some earlier versions of TLS. It MUST NOT be sent by compliant implementations. |
protocol_version | The protocol version the client has attempted to negotiate is recognized but not supported. (For example, old protocol versions might be avoided for security reasons.) This message is always fatal. |
insufficient_security | Returned instead of handshake_failure when a negotiation has failed specifically because the server requires ciphers more secure than those supported by the client. This message is always fatal. |
internal_error | An internal error unrelated to the peer or the correctness of the protocol (such as a memory allocation failure) makes it impossible to continue. This message is always fatal. |
user_canceled | This handshake is being canceled for some reason unrelated to a protocol failure. If the user cancels an operation after the handshake is complete, just closing the connection by sending a close_notify is more appropriate. This alert should be followed by a close_notify. This message is generally a warning. |
no_renegotiation | Sent by the client in response to a hello request or by the server in response to a client hello after initial handshaking. Either of these would normally lead to renegotiation; when that is not appropriate, the recipient should respond with this alert. At that point, the original requester can decide whether to proceed with the connection. One case where this would be appropriate is where a server has spawned a process to satisfy a request; the process might receive security parameters (key length, authentication, etc.) at startup, and it might be difficult to communicate changes to these parameters after that point. This message is always a warning. |
unsupported_extension | sent by clients that receive an extended server hello containing an extension that they did not put in the corresponding client hello. This message is always fatal. |
Related Topics
Spam Blocker
Spam Blocker is an intelligent email filter that identifies and handles spam (unsolicited bulk email). It leverages technology from the SpamAssassin project and improves upon itby integrating a commercial spam engine. It can scan any email that is transported via SMTP.
Spam Blocker transparently scans email transported over SMTP on the way to your mail server (or outbound if configured so). It does not require reconfiguring your DNS MX records or the email server. Any SMTP traffic going through the NG Firewall server will be scanned.
Settings
This section reviews the different settings and configuration options available for Spam Blocker.
Status
![](/assets/images/user-manual/um-etm-me/Graphics/Spam%20Blocker/apps_spam-blocker_status.png)
- Scan SMTP: This enables or disables SMTP scanning.
- Strength: If the Spam Score of a message is equal to or greater than this setting, your chosen action will be taken regarding the message. Higher values make Spam Blocker more sensitive to spam.
- Action: The action was taken on the message if the Spam Score is high enough.
- If set to Mark, "[Spam]..." will be prepended to the email subject line and delivered. If set to Pass, the message will be delivered as originally sent. Drop will inform the sending server the mail was successfully delivered, but NG Firewall will drop the mail so it is never delivered. Quarantine will send the mail to users' email quarantine for them to release or delete as they see fit. For more information, refer to Quarantine.
- Drop Super Spam: If this option is enabled, any emails with a score greater than the Super spam score will be dropped.
- Super Spam Score: The score emails must reach to be dropped as Super Spam.
- Advanced SMTP Configuration
-
Important: The default values are the suggested values. Changing and customizing settings can easily and will likely cause Spam Blocker to perform less than optimally.
-
- Enable tarpitting: This option enables Tarpit. If enabled, when an SMTP session is first caught Spam Blocker will check if the client IP is on a Domain Name System BLocklist (DNSBL). If it is, the session is rejected before the remote server is even allowed to send the email. This increases the capacity of a given server by quite a bit and can also save bandwidth, but it can increase false positives if the remote email server has mistakenly been put on a blacklist. This setting will not increase spam scanning accuracy - it will actually decrease it as it will prevent valuable super-spam training data from reaching the spam engine. By enabling this feature, you are getting lower spam accuracy but increased email scanning capacity.
- Enable greylisting: This option enables greylisting. If enabled, each time a new sender tries to send mail to a specific receiver, it will receive a "421 Please try again" error. The second time, mail will be allowed. Greylisting will reduce spam because spammers often won't retry transmissions as they should, or the extra time delay will increase the chances of the spam engine properly identifying new spam waves. However, greylisting adds a delay to all legitimate email. This settings is not suggested for most sites because of the complications.
- Add email headers: When enabled, NG Firewall adds information about the Spam Score and the test run to get that score to the headers of the message.
- Close connection on scan failure: This option will close the connection if the scan fails, so the message will be resent and retested. If disabled, a scan failure will allow the email to be delivered without being scanned.
- Scan outbound (WAN) SMTP: If unchecked, outbound mail (mail in sessions going out a WAN interface) will not be scanned. If checked, outbound mail will be scanned just like incoming mail.
- Allow and ignore TLS sessions: This option controls the allowance of TLS sessions. If unchecked (the default), the TLS advertisement (if present) is removed from the server advertisements and TLS is not allowed on any scanned sessions. If checked, the TLS advertisement is allowed and if the client initialized TLS the message will pass through completely unscanned, even if it is spam.
- CPU Load Limit: If your CPU Load exceeds this number incoming connections are stopped until the load decreases. This is specified so that spam scanning can not monopolize the server resources.
- Concurrent Scan Limit: This is the maximum number of messages that can be scanned at the same time. This is specified so that spam scanning can not monopolize the server resources.
- Message Size Limit: This option allows you to change the maximum size of a message that will be scanned for spam. The default maximum size is 256KB. Spam will typically be much smaller, as spammers rely on the sheer number of messages sent.
Note: This does not control the message size limit of messages passed through NG Firewall. This does not effect the maximum size of the message your server will accept, only the limit on the size of message that will be checked for spam.
-
Reporting
The Reports tab provides a view of all reports and events for all traffic handled by Spam Blocker.
Reports
This applications reports can be accessed via the Reports tab at the top or the Reports tab within the settings. All pre-defined reports will be listed along with any custom reports that have been created.
Reports can be searched and further defined using the time selectors and the Conditions window at the bottom of the page. The data used in the report can be obtained on the Current Data window on the right.
Report Entry | Description |
---|---|
Spam Blocker Summary | A summary of spam blocking actions for email activity. |
Email Usage (all) | The amount of scanned, clean, and spam email over time. |
Email Usage (scanned) | The amount of scanned email over time. |
Email Usage (clean) | The amount of clean email over time. |
Email Usage (spam) | The amount of spam email over time. |
Spam Ratio | The ratio of spam (true) to ham (false) |
Top Spam Recipients | The number of email addresses with spam. |
Top Spam Sender Addresses | The number of IP addresses sending spam. |
All Email Events | All emails are scanned by Spam Blocker. |
All Spam Events | All emails are marked as Spam. |
Quarantined Events | All emails marked as Spam and quarantined. |
Tarpit Events | All email sessions that were tarpitted. |
Spam Blocker Reports
The Reports tab provides a view of all reports and events for all traffic handled by Spam Blocker.
Reports
This applications reports can be accessed via the Reports tab at the top or the Reports tab within the settings. All pre-defined reports will be listed along with any custom reports that have been created.
Reports can be searched and further defined using the time selectors and the Conditions window at the bottom of the page. The data used in the report can be obtained on the Current Data window on the right.
Report Entry | Description |
---|---|
Spam Blocker Summary | A summary of spam blocking actions for email activity. |
Email Usage (all) | The amount of scanned, clean, and spam email over time. |
Email Usage (scanned) | The amount of scanned email over time. |
Email Usage (clean) | The amount of clean email over time. |
Email Usage (spam) | The amount of spam email over time. |
Spam Ratio | The ratio of spam (true) to ham (false) |
Top Spam Recipients | The number of email addresses with spam. |
Top Spam Sender Addresses | The number of IP addresses sending spam. |
All Email Events | All emails are scanned by Spam Blocker. |
All Spam Events | All emails are marked as Spam. |
Quarantined Events | All emails are marked as Spam and quarantined. |
Tarpit Events | All email sessions that were tarpitted. |
Web Filter
Web Filter monitors HTTP and HTTPS traffic on your network to filter and log web activities and block inappropriate content. Web Filter also appeals to customers who require an added level of protection or are subject to regulations, for example Web Filter helps libraries comply with the Children's Internet Protection Act). Need to block Pornography or Hate Speech on your network? Web Filter is your answer.
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Filter/WebFilter.png)
About Web Filter
- Real-time classification and updates: When your users visit a site, NG Firewall sends the URL to the Webroot BrightCloud® to be categorized. When the data is returned, NG Firewall keeps a temporary local cache of the site and category to speed up the process the next time the URL is requested. This data is then used to flag or allow users access to the site they have requested, all without any appreciable increase in load time. If a site is not categorized upon request, it is autocategorized by our partners at Webroot and put into a queue to be verified by a human. Because this is done dynamically, new sites and updated URLs are allowed or flagged according to your settings without additional intervention, plus you have the option of requesting recategorization of sites.
- HTTPS Filtering: Web Filter has multiple techniques to deal with HTTPS, SSL-encrypted HTTP. HTTPS traffic is encrypted so only some information is visible and this information is used to categorize the session. More information on how this is down below.
- Detailed categorization: Web Filter offers 79 categories and tens of billions of URLs. The Web Filter database is over 100 times larger and more accurate. The abundance of categories means that you can narrow your scope - maybe you want to flag websites related to nudity, but allow sites dealing with Sexual Education.
- Advanced features: Force safe-search on search engines, filter and log user searches, restrict google domains, and more!
Traffic Flow
When scanning traffic, Web Filter evaluates the pass lists, block lists, categories, and rules at two distinct points of the HTTP transaction. The first evaluation happens after the request is received from the client and before it is forwarded to the server. The second is after the response is received from the server and before it is passed back to the client. This allows a high degree of filtering and control over both resources that are requested, and content that is returned in response.
HTTP Request
- A lookup is performed to determine the category for the requested site. The category is attached to the session for use by Web Filter as well as other applications.
- The source IP of the request is checked against the Pass Clients list. If a match is found, the traffic is allowed.
- The destination site of the request is checked against the Pass Sites list. If a match is found, the traffic is allowed.
- If 'Restrict Google applications' is enabled, the appropriate header is added to the request using the 'Allowed Domains' that have been configured.
- If 'Pass if referrer matches any Pass Sites' is enabled, the referrer is checked against the Pass Sites list. If a match is found, the traffic is allowed.
- If the Unblock option is enabled, the destination site and the source IP of the request are checked against the unblock list. If a match is found, the traffic is allowed.
- If 'Block pages from IP only hosts' is enabled, the request will be evaluated and blocked if the destination is an IP address.
- The destination site of the request is checked against the Block Sites list. If a match is found, the traffic is blocked.
- The traffic details are passed to the Rules list. If a match is found, the traffic is allowed, flagged, or blocked based on the options configured in the rule that was matched.
- The category determined in Step 1 is compared to the Categories list, and the traffic is allowed, flagged, or blocked based on the corresponding match. If the category could not be determined, the traffic is allowed.
HTTP Response
- The source IP of the request is checked against the Pass Clients list. If a match is found, the traffic is allowed.
- The site from which the response was received is checked against the Pass Sites list. If a match is found, the traffic is allowed.
- If the Unblock option is enabled, the site from which the response was received and the client IP are checked against the unblock list. If a match is found, the traffic is allowed.
- The traffic details are passed to the Rules list. If a match is found, the traffic is allowed, flagged, or blocked based on the options configured in the rule that was matched.
Settings
This section reviews the different settings and configuration options available for Web Filter.
Status
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Filter/apps_web-filter_status.png)
Categories
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Filter/apps_web-filter_categories.png)
Search Terms
Search Terms filtering enables you to flag or block specific search terms your users perform on popular search sites including Google, Bing, Ask, Yahoo, and YouTube. For example, if someone searches Google and includes the word "suicide", or they search for "twerking" videos on YouTube you can have these activities flagged or blocked.
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Filter/apps_web-filter_search-terms.png)
Site Lookup
Site Lookup allows you to find the categorization of a URL. Clicking it brings up a dialog. In Site URL specify the URL to find and click Search to find the URL's categorization.
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Filter/apps_web-filter_site-lookup.png)
Block Sites
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Filter/apps_web-filter_block-sites.png)
Pass Sites
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Filter/apps_web-filter_pass-sites.png)
Pass Clients
If you add an IP address to this list, Web Filter will not block any traffic from that IP regardless of the blocked categories or sites. Just add the IP and save. Unchecking the pass option will have the block/pass lists affect the user as if they were not entered into the Passed Client IPs list. This list uses IP Matcher syntax.
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Filter/apps_web-filter_pass-clients.png)
Rules
The Rules tab allows you to specify rules to Block or Flag traffic that passes through Web Filter.
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Filter/apps_web-filter_rules.png)
- Flag: Allows the traffic which matched the rule to flow, and flags the traffic for easier viewing in the event log.
- Block: Blocks the traffic which matched the rule.
Rule Types
In previous versions of Web Filter, there were dedicated lists for blocking certain file extensions or MIME types. This capability is still available using the more flexible filter rules. For blocking specific file extensions, you can create a rule with the condition HTTP: Response File Extension that has a comma separated list of the extensions to block in the Value field. For blocking MIME types, you would create a rule with the condition HTTP: Content Type that has a comma separated list of the content types to block in the Value field.
Extension | Category | Description |
---|---|---|
exe | executable | an executable file format |
ocx | executable | an executable file format |
dll | executable | an executable file format |
cab | executable | an ActiveX executable file format |
bin | executable | an executable file format |
com | executable | an executable file format |
jpg | image | an image file format |
png | image | an image file format |
gif | image | an image file format |
jar | java | a Java file format |
class | java | a Java file format |
swf | flash | the flash file format |
mp3 | audio | an audio file format |
wav | audio | an audio file format |
wmf | audio | an audio file format |
mpg | video | a video file format |
mov | video | a video file format |
avi | video | a video file format |
hqx | archive | an archived file format |
cpt | compression | a compressed file format |
Content | Category | Description |
---|---|---|
application/octet-stream | unspecified data | byte stream |
application/x-msdownload | Microsoft download | executable |
application/exe | executable | executable |
application/x-exe | executable | executable |
application/dos-exe | DOS executable | executable |
application/x-winexe | Windows executable | executable |
application/msdos-windows | MS-DOS executable | executable |
application/x-msdos-program | MS-DOS program | executable |
application/x-oleobject | Microsoft OLE Object | executable |
application/x-java-applet | Java Applet | executable |
audio/mpegurl | MPEG audio URLs | audio |
audio/x-mpegurl | MPEG audio URLs | audio |
audio/mp3 | MP3 audio | audio |
audio/x-mp3 | MP3 audio | audio |
audio/mpeg | MPEG audio | audio |
audio/mpg | MPEG audio | audio |
audio/x-mpeg | MPEG audio | audio |
audio/x-mpg | MPEG audio | audio |
application/x-ogg | Ogg Vorbis | audio |
audio/m4a | MPEG 4 audio | audio |
audio/mp2 | MP2 audio | audio |
audio/mp1 | MP1 audio | audio |
application/ogg | Ogg Vorbis | audio |
audio/wav | Microsoft WAV | audio |
audio/x-wav | Microsoft WAV | audio |
audio/x-pn-wav | Microsoft WAV | audio |
audio/aac | Advanced Audio Coding | audio |
audio/midi | MIDI audio | audio |
audio/mpeg | MPEG audio | audio |
audio/aiff | AIFF audio | audio |
audio/x-aiff | AIFF audio | audio |
audio/x-pn-aiff | AIFF audio | audio |
audio/x-pn-windows-acm | Windows ACM | audio |
audio/x-pn-windows-pcm | Windows PCM | audio |
audio/basic | 8-bit u-law PCM | audio |
audio/x-pn-au | Sun audio | audio |
audio/3gpp | 3GPP | audio |
audio/3gpp-encrypted | encrypted 3GPP | audio |
audio/scpls | streaming mp3 playlists | audio |
audio/x-scpls | streaming mp3 playlists | audio |
application/smil | SMIL | audio |
application/sdp | Streaming Download Project | audio |
application/x-sdp | Streaming Download Project | audio |
audio/amr | AMR codec | audio |
audio/amr-encrypted | AMR encrypted codec | audio |
audio/amr-wb | AMR-WB codec | audio |
audio/amr-wb-encrypted | AMR-WB encrypted codec | audio |
audio/x-rn-3gpp-amr | 3GPP codec | audio |
audio/x-rn-3gpp-amr-encrypted | 3GPP-AMR encrypted codec | audio |
audio/x-rn-3gpp-amr-wb | 3gpp-AMR-WB codec | audio |
audio/x-rn-3gpp-amr-wb-encrypted | 3gpp-AMR_WB encrypted codec | audio |
application/streamingmedia | Streaming Media | audio |
video/mpeg | MPEG video | video |
audio/x-ms-wma | Windows Media | video |
video/quicktime | QuickTime | video |
video/x-ms-asf | Microsoft ASF | video |
video/x-msvideo | Microsoft AVI | video |
video/x-sgi-mov | SGI movie | video |
video/3gpp | 3GPP video | video |
video/3gpp-encrypted | 3GPP encrypted video | video |
video/3gpp2 | 3GPP2 video | video |
audio/x-realaudio | RealAudio | audio |
text/vnd.rn-realtext | RealText | text |
audio/vnd.rn-realaudio | RealAudio | audio |
audio/x-pn-realaudio | RealAudio plug-in | audio |
image/vnd.rn-realpix | RealPix | image |
application/vnd.rn-realmedia | RealMedia | video |
application/vnd.rn-realmedia-vbr | RealMedia VBR | video |
application/vnd.rn-realmedia-secure | secure RealMedia | video |
application/vnd.rn-realaudio-secure | secure RealAudio | audio |
audio/x-realaudio-secure | secure RealAudio | audio |
video/vnd.rn-realvideo-secure | secure RealVideo | video |
video/vnd.rn-realvideo | RealVideo | video |
application/vnd.rn-realsystem-rmj | RealSystem media | video |
application/vnd.rn-realsystem-rmx | RealSystem secure media | video |
audio/rn-mpeg | MPEG audio | audio |
application/x-shockwave-flash | Macromedia Shockwave | multimedia |
application/x-director | Macromedia Shockwave | multimedia |
application/x-authorware-bin | Macromedia Authorware binary | multimedia |
application/x-authorware-map | Macromedia Authorware shocked file | multimedia |
application/x-authorware-seg | Macromedia Authorware shocked packet | multimedia |
application/futuresplash | Macromedia FutureSplash | multimedia |
application/zip | ZIP | archive |
application/x-lzh | LZH archive | archive |
image/gif | Graphics Interchange Format | image |
image/png | Portable Network Graphics | image |
image/jpeg | JPEG | image |
image/bmp | Microsoft BMP | image |
image/tiff | Tagged Image File Format | image |
image/x-freehand | Macromedia Freehand | image |
image/x-cmu-raster | CMU Raster | image |
image/x-rgb | RGB image | image |
text/css | cascading style sheet | text |
text/html | HTML | text |
text/plain | plain text | text |
text/richtext | rich text | text |
text/tab-separated-values | tab separated values | text |
text/xml | XML | text |
text/xsl | XSL | text |
text/x-sgml | SGML | text |
text/x-vcard | vCard | text |
application/mac-binhex40 | Macintosh BinHex | archive |
application/x-stuffit | Macintosh Stuffit archive | archive |
application/macwriteii | MacWrite Document | document |
application/applefile | Macintosh File | archive |
application/mac-compactpro | Macintosh Compact Pro | archive |
application/x-bzip2 | block compressed | compressed |
application/x-shar | shell archive | archive |
application/x-gtar | gzipped tar archive | archive |
application/x-gzip | gzip compressed | compressed |
application/x-tar | 4.3BSD tar archive | archive |
application/x-ustar | POSIX tar archive | archive |
application/x-cpio | old cpio archive | archive |
application/x-bcpio | POSIX cpio archive | archive |
application/x-sv4crc | System V cpio with CRC | archive |
application/x-compress | UNIX compressed | compressed |
application/x-sv4cpio | System V cpio | archive |
application/x-sh | UNIX shell script | executable |
application/x-csh | UNIX csh script | executable |
application/x-tcl | Tcl script | executable |
application/x-javascript | JavaScript | executable |
application/x-excel | Microsoft Excel | document |
application/mspowerpoint | Microsoft Powerpoint | document |
application/msword | Microsoft Word | document |
application/wordperfect5.1 | Word Perfect | document |
application/rtf | Rich Text Format | document |
application/pdf | Adobe Acrobat | document |
application/postscript | Postscript | document |
Advanced
The Advanced section enables you to configure additional web filter options.
- Process HTTPS traffic by SNI (Server Name Indication) if present: If this option is enabled, HTTPS traffic will be categorized using the "Server Name Indication" in the HTTPS data stream, if present. More details in HTTPS Options.
- Process HTTPS traffic by hostname in server certificate when SNI information not present: If this option is enabled and SNI information is not present, the certificate is fetched from the HTTPS server and the server name on the certificate will be used for categorization and filtering purposes.
- Process HTTPS traffic by server IP if both SNI and certificate hostname information are not available: If this option is enabled and neither of the previous options worked, HTTPS traffic will be categorized using the IP address. More details in HTTPS Options.
- Enforce safe search on popular search engines: When this option is enabled, safe search will be enforced on all searches using supported search engines: Google, Yahoo!, Bing, Ask.
- Enforce restrict mode on YouTube: When this option is enabled, restrict mode will be enforced on all YouTube content.
- Force searches through kid-friendly search engine: When this option is enabled, all searches in popular search engines will be redirected through kidzsearch.com. kidzsearch is a visual child-safe search engine and web portal powered by Google Custom Search with academic autocomplete that emphasizes safety for children.
- Block QUIC Sessions (UDP port 443): If enabled, Web Filter prevents browsers from using the QUIC protocol. QUIC is used by Chrome browser to access many types of Google applications and services. By allowing QUIC, NG Firewall has less visibility and control over this type of traffic.
- Block pages from IP only hosts: When this option is enabled, users entering an IP address rather than domain name will be blocked.
- Pass if referrer matches any Pass Sites. When this option is checked, if a site allowed via a Pass Site entry links to external content using a referrer, that external content will be passed regardless of Category settings. Please note that this setting requires SSL Inspector to work with HTTPS sites and many sites no longer use referrers.
- Close connection for blocked HTTPS sessions without redirecting to block page. If enabled, secure sites blocked by Web Filter do not redirect the user to a denial page and close the connection without any notice to the user. This is useful when you are not using SSL Inspector and the server's root certificate authority is not installed on the client device.
- Restrict Google applications: When this option is enabled, only domains listed in Domain are allowed to access Google applications such as Gmail. All others are blocked by Google. Multiple domains can be specified, separated by commas such as google.com,domain.com. This adds an X-GoogApps-Allowed-Domains header to web requests which then gets enforced on Google's servers. More information on this feature can be found here.
Custom Block Page
- Custom block page URL: Set an external location where to redirect users when they are denied access to a web site by Web Filter. This is useful if you want to use your own server to process the denial in a different way than the built-in denial options.
Variable | Description |
---|---|
reason | The reason the user was denied access. |
appname | The NG Firewall app that is responsible for the denial. |
appid | The ID of the NG Firewall app that is responsible for the denial. |
clientAddress | The IP Address of the denied device. |
url | The denied URL that the user requested. |
- Unblock: This section can be used to add a button to allow users to bypass restrictions on a case-by-case basis.
- If Unblock is set to None, no users will be allowed to bypass the block page. If Unblock is set to Temporary, users will be allowed to visit the site for one hour from the time it is unblocked. If Unblock is set to Permanent and Global, then users will be allowed to visit the site and unblocked sites will be added to the permanent global pass list so it will always be allowed in the future.
- Clear Category URL Cache: This option will clear the local cache of categorized sites and URLs. After clearing the cache all new web visits will be looked up fresh using the categorization service. The cache automatically cleans itself as entries become old or stale, so this is mostly for testing.
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Filter/apps_web-filter_advanced.png)
Reports
The Reports tab provides a view of all reports and events for all traffic handled by Web Filter.
This applications reports can be accessed via the Reports tab at the top or the Reports tab within the settings. All pre-defined reports will be listed along with any custom reports that have been created.
Reports can be searched and further defined using the time selectors and the Conditions window at the bottom of the page. The data used in the report can be obtained on the Current Data window on the right.
Report Entry | Description |
---|---|
Web Filter Summary | A summary of web filter actions. |
Web Usage | The amount of total, flagged, and blocked web requests over time. |
Web Usage (scanned) | The amount of total, flagged, and blocked web requests over time. |
Web Usage (flagged) | The amount of flagged, and blocked web requests over time. |
Web Usage (blocked) | The amount of flagged, and blocked web requests over time. |
Top Categories (by request) | The number of web requests grouped by category. |
Top Categories (by size) | The sum of the size of requested web content grouped by category. |
Top Flagged Categories | The number of flagged web requests grouped by category. |
Top Blocked Categories | The number of blocked web requests grouped by category. |
Top Sites (by request) | The number of web requests grouped by website. |
Top Sites (by size) | The sum of the size of requested web content grouped by website. |
Top Flagged Sites | The number of flagged web requests grouped by website. |
Top Blocked Sites | The number of blocked web requests grouped by website. |
Top Domains (by request) | The number of web requests grouped by domain. |
Top Domains (by size) | The sum of the size of requested web content grouped by domain. |
Top Flagged Domains | The number of flagged web requests grouped by domain. |
Top Blocked Domains | The number of blocked web requests grouped by domain. |
Top Domains Usage | The amount of web requests per top domain. |
Top Hostnames (by requests) | The number of web requests grouped by hostname. |
Top Hostnames (by size) | The sum of the size of requested web content grouped by hostname. |
Top Flagged Hostnames | The number of flagged web request grouped by hostname. |
Top Blocked Hostnames | The number of blocked web request grouped by hostname. |
Top Clients (by requests) | The number of web requests grouped by client. |
Top Clients (by size) | The sum of the size of requested web content grouped by client. |
Top Flagged Clients | The number of flagged web request grouped by client. |
Top Blocked Clients | The number of blocked web request grouped by client. |
Top Usernames (by requests) | The number of web requests grouped by username. |
Top Usernames (by size) | The sum of the size of requested web content grouped by username. |
Top Flagged Usernames | The number of flagged web request grouped by username. |
Top Blocked Usernames | The number of blocked web request grouped by username. |
Top Content (by request) | The number of web requests grouped by category. |
Top Content (by size) | The sum of the size of requested web content grouped by category. |
Top Searches | The number of non blocked, non-flagged search queries grouped by term. |
Top Flagged Searches | The number of flagged search queries grouped by term. |
Top Blocked Searches | The number of blocked search queries grouped by term. |
All Web Events | Shows all scanned web requests. |
Flagged Web Events | Shows all flagged web requests. |
Blocked Web Events | Shows all blocked web requests. |
All HTTP Events | Shows all scanned unencrypted HTTP requests. |
All HTTPS Events | Shows all encrypted HTTPS requests. |
Unblocked Web Events | Shows all unblocked web requests |
All Search Events | Shows all search queries processed by Web Filter. |
Flagged Search Events | Shows flagged search queries processed by Web Filter. |
Blocked Search Events | Shows blocked search queries processed by Web Filter. |
The tables queried to render these reports:
HTTPS Options
There are many ways to handle HTTPS. An overview of the various techniques is described here.
If SSL Inspector is installed and inspects a session, then it is fully decrypted to HTTP before Web Filter processes the session. In this case HTTPS is treated identically to HTTP. If SSL Inspector is not installed or the session is not inspected, there are still several techniques to handle encrypted HTTP sessions.
- Process HTTPS traffic by SNI (Server Name Indication) if present.
- Process HTTPS traffic by hostname in server certificate when SNI information not present
- Process HTTPS traffic by server IP if both SNI and certificate hostname information are not available.
If Process HTTPS traffic by SNI (Server Name Indication) if present encrypted port-443 traffic will be scanned. Most modern browsers on modern OSs will send the hostname of the server in cleartext - this is called "Server Name Indication" or SNI. SNI is an optional cleartext field in the HTTPS request that shows the hostname of the server. If this option is enabled and the SNI information is present in the HTTPS request, this hostname will be used as the URL for this request and all categorization, flag lists, and pass lists, will be processed as if this were a regular HTTP request to that URL.
If the SNI-based categorization determines the page should be passed (and/or flagged) then the session is allowed and the appropriate event based on the SNI information is logged ("https://example.com/").
For example, if the user visits "https://wellsfargo.com/welcome" in the browser, "wellsfargo.com" is seen as the SNI information. If SNI-based categorization is enabled, the request will be handled exactly like "http://wellsfargo.com" would be. If Web Filter is configured to flag "Financial Services" , then "https://wellsfargo.com/welcome" will be flagged, unless "wellsfargo.com" is in the pass list or the client IP address is in the client IP pass list.
If No SNI information is present and Process HTTPS traffic by hostname in server certificate when SNI information not present is enabled, then the hostname will be pulled from the certificate presented to the client.
For example, if the user visits "https://wellsfargo.com/welcome" in a non-SNI enabled browser, then there is no SNI information. In this case if Process HTTPS traffic by hostname in server certificate when SNI information not present is enabled it will use the certificate information instead to categorize the session. It will download the certificate from the site and see that the certificate is "Issued To" "www.wellsfargo.com." It will use this information to check the category for "https://www.wellsfargo.com" and categorize the session.
If no SNI or certificate information was available and Process HTTPS traffic by server IP if both SNI and certificate hostname information are not available the session will be processed and categorized by IP address. If the IP-based processing and categorization of the web requests determines the session should be flagged, the session is reset and no more processing of this session will be done. If the IP-based processing and categorization determines the page should be passed (and/or flagged) then the session is allowed and the appropriate event based on its IP is logged ("https://1.2.3.4").
For example, if the user visits "https://wellsfargo.com/welcome" in a non-SNI enabled browser, then there is no SNI information. If the the certificate information was missing for some reason then this session can only be identified by IP address. In this case if Process HTTPS traffic by server IP if both SNI and certificate hostname information are not available is enabled it will use the IP address instead. So it will process/categorize this web request as 'http://1.2.3.4' if 1.2.3.4 is the IP of wellsfargo.com. This will still often result in correct categorization for dedicated web servers, but does poorly when using generic cloud computing servers that offer a wide variety of websites.
To see the HTTPS categorization in action use the "All HTTPS Events" query in the event log.
Web Filter Reports
The Reports tab provides a view of all reports and events for all traffic handled by Web Filter.
Reports
This applications reports can be accessed via the Reports tab at the top or the Reports tab within the settings. All pre-defined reports will be listed along with any custom reports that have been created.
Reports can be searched and further defined using the time selectors and the Conditions window at the bottom of the page. The data used in the report can be obtained on the Current Data window on the right.
Report Entry | Description |
---|---|
Web Filter Summary | A summary of web filter actions. |
Web Usage | The amount of total, flagged, and blocked web requests over time. |
Web Usage (scanned) | The amount of total, flagged, and blocked web requests over time. |
Web Usage (flagged) | The amount of flagged, and blocked web requests over time. |
Web Usage (blocked) | The amount of flagged, and blocked web requests over time. |
Top Categories (by request) | The number of web requests grouped by category. |
Top Categories (by size) | The sum of the size of requested web content grouped by category. |
Top Flagged Categories | The number of flagged web requests grouped by category. |
Top Blocked Categories | The number of blocked web requests grouped by category. |
Top Sites (by request) | The number of web requests grouped by website. |
Top Sites (by size) | The sum of the size of requested web content grouped by website. |
Top Flagged Sites | The number of flagged web requests grouped by website. |
Top Blocked Sites | The number of blocked web requests grouped by website. |
Top Domains (by request) | The number of web requests grouped by domain. |
Top Domains (by size) | The sum of the size of requested web content grouped by domain. |
Top Flagged Domains | The number of flagged web requests grouped by domain. |
Top Blocked Domains | The number of blocked web requests grouped by domain. |
Top Domains Usage | The amount of web requests per top domain. |
Top Hostnames (by requests) | The number of web requests grouped by hostname. |
Top Hostnames (by size) | The sum of the size of requested web content grouped by hostname. |
Top Flagged Hostnames | The number of flagged web request grouped by hostname. |
Top Blocked Hostnames | The number of blocked web request grouped by hostname. |
Top Clients (by requests) | The number of web requests grouped by client. |
Top Clients (by size) | The sum of the size of requested web content grouped by client. |
Top Flagged Clients | The number of flagged web request grouped by client. |
Top Blocked Clients | The number of blocked web request grouped by client. |
Top Usernames (by requests) | The number of web requests grouped by username. |
Top Usernames (by size) | The sum of the size of requested web content grouped by username. |
Top Flagged Usernames | The number of flagged web request grouped by username. |
Top Blocked Usernames | The number of blocked web request grouped by username. |
Top Content (by request) | The number of web requests grouped by category. |
Top Content (by size) | The sum of the size of requested web content grouped by category. |
Top Searches | The number of non blocked, non-flagged search queries grouped by term. |
Top Flagged Searches | The number of flagged search queries grouped by term. |
Top Blocked Searches | The number of blocked search queries grouped by term. |
All Web Events | Shows all scanned web requests. |
Flagged Web Events | Shows all flagged web requests. |
Blocked Web Events | Shows all blocked web requests. |
All HTTP Events | Shows all scanned unencrypted HTTP requests. |
All HTTPS Events | Shows all encrypted HTTPS requests. |
Unblocked Web Events | Shows all unblocked web requests |
All Search Events | Shows all search queries processed by Web Filter. |
Flagged Search Events | Shows flagged search queries processed by Web Filter. |
Blocked Search Events | Shows blocked search queries processed by Web Filter. |
Related Topics
Web Monitor
Web Monitor monitors HTTP and HTTPS traffic on your network to log web activities and flag inappropriate content.
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Monitor/WebMonitor.png)
About Web Monitor
- Real-time classification and updates: When your users visit a site, NG Firewall sends the URL to the Webroot BrightCloud® to be categorized. When the data is returned, NG Firewall keeps a temporary local cache of the site and category to speed up the process the next time the URL is requested. This data is then used to flag or allow users access to the site they have requested, all without any appreciable increase in load time. If a site is not categorized upon request, it is autocategorized by our partners at Webroot and put into a queue to be verified by a human. Because this is done dynamically, new sites and updated URLs are allowed or flagged according to your settings without additional intervention, plus you have the option of requesting recategorization of sites.
- HTTPS Filtering: Web Monitor has multiple techniques to deal with HTTPS, SSL-encrypted HTTP. HTTPS traffic is encrypted so only some information is visible and this information is used to categorize the session. More information on how this is down below.
- Detailed categorization: Web Monitor offers 79 categories and tens of billions of URLs. The Web Monitor database is over 100 times larger and more accurate. The abundance of categories means that you can narrow your scope - maybe you want to flag websites related to nudity, but allow sites dealing with Sexual Education.
Traffic Flow
When scanning traffic, Web Monitor evaluates the pass lists, flag lists, categories, and rules at two distinct points of the HTTP transaction. The first evaluation happens after the request is received from the client and before it is forwarded to the server. The second is after the response is received from the server and before it is passed back to the client. This allows a high degree of monitoring over both resources that are requested, and content that is returned in response.
HTTP Request
- A lookup is performed to determine the category for the requested site. The category is attached to the session for use by Web Monitor as well as other applications.
- The source IP of the request is checked against the Pass Clients list. If a match is found, the traffic is allowed.
- The destination site of the request is checked against the Pass Sites list. If a match is found, the traffic is allowed.
- The destination site of the request is checked against the Flag Sites list. If a match is found, the traffic is flagged.
- The traffic details are passed to the Rules list. If a match is found, the traffic is allowed and possibly flagged based on the options configured in the rule that was matched.
- The category determined in step #1 is compared to the Categories list, and the traffic is allowed and possibly flagged based on the corresponding match. If the category could not be determined, the traffic is allowed.
HTTP Response
- The source IP of the request is checked against the Pass Clients list. If a match is found, the traffic is allowed.
- The site from which the response was received is checked against the Pass Sites list. If a match is found, the traffic is allowed.
- The traffic details are passed to the Rules list. If a match is found, the traffic is allowed and possibly flagged based on the options configured in the rule that was matched.
Settings
This section reviews the different settings and configuration options available for Web Monitor.
Status
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Monitor/apps_web-monitor_status.png)
Categories
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Monitor/apps_web-monitor_categories.png)
Flag Sites
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Monitor/apps_web-monitor_flag-sites.png)
Pass Sites
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Monitor/apps_web-monitor_pass-sites.png)
Pass Clients
If you add an IP address to this list, Web Monitor will not flag any traffic from that IP regardless of the flagged categories or sites. Just add the IP and save. Unchecking the pass option will have the flag/pass lists affect the user as if they were not entered into the Passed Client IPs list. This list uses IP Matcher syntax.
- If you have a few users that need to completely bypass Web Monitor controls, consider using pass lists. If you have users that simply need different Web Monitor settings, you should set up a separate rack using Policy Manager. When using this feature, please remember that DHCP IPs can change, so you'll probably want to set up either a Static IP or a Static DHCP Lease for the machine in question.
Figure 20. Web Monitor Pass Clients
Rules
The Rules tab allows you to specify rules to Flag traffic that passes through Web Monitor.
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Monitor/apps_web-monitor_rules.png)
- Flag: Allows the traffic which matched the rule to flow, and flags the traffic for easier viewing in the event log.
Rule Types
In previous versions of Web Monitor, there were dedicated lists for flagging certain file extensions or MIME types. This capability is still available using the more flexible filter rules. For flagging specific file extensions, you can create a rule with the condition Web Filter: Response File Extension that has a comma separated list of the extensions to flag in the Value field. For flagging MIME types, you would create a rule with the condition Web Filter: Response Content Type that has a comma separated list of the content types to flag in the Value field.
Extension | Category | Description |
---|---|---|
exe | executable | an executable file format |
ocx | executable | an executable file format |
dll | executable | an executable file format |
cab | executable | an ActiveX executable file format |
bin | executable | an executable file format |
com | executable | an executable file format |
jpg | image | an image file format |
png | image | an image file format |
gif | image | an image file format |
jar | java | a Java file format |
class | java | a Java file format |
swf | flash | the flash file format |
mp3 | audio | an audio file format |
wav | audio | an audio file format |
wmf | audio | an audio file format |
mpg | video | a video file format |
mov | video | a video file format |
avi | video | a video file format |
hqx | archive | an archived file format |
cpt | compression | a compressed file format |
Content | Category | Description |
---|---|---|
application/octet-stream | unspecified data | byte stream |
application/x-msdownload | Microsoft download | executable |
application/exe | executable | executable |
application/x-exe | executable | executable |
application/dos-exe | DOS executable | executable |
application/x-winexe | Windows executable | executable |
application/msdos-windows | MS-DOS executable | executable |
application/x-msdos-program | MS-DOS program | executable |
application/x-oleobject | Microsoft OLE Object | executable |
application/x-java-applet | Java Applet | executable |
audio/mpegurl | MPEG audio URLs | audio |
audio/x-mpegurl | MPEG audio URLs | audio |
audio/mp3 | MP3 audio | audio |
audio/x-mp3 | MP3 audio | audio |
audio/mpeg | MPEG audio | audio |
audio/mpg | MPEG audio | audio |
audio/x-mpeg | MPEG audio | audio |
audio/x-mpg | MPEG audio | audio |
application/x-ogg | Ogg Vorbis | audio |
audio/m4a | MPEG 4 audio | audio |
audio/mp2 | MP2 audio | audio |
audio/mp1 | MP1 audio | audio |
application/ogg | Ogg Vorbis | audio |
audio/wav | Microsoft WAV | audio |
audio/x-wav | Microsoft WAV | audio |
audio/x-pn-wav | Microsoft WAV | audio |
audio/aac | Advanced Audio Coding | audio |
audio/midi | MIDI audio | audio |
audio/mpeg | MPEG audio | audio |
audio/aiff | AIFF audio | audio |
audio/x-aiff | AIFF audio | audio |
audio/x-pn-aiff | AIFF audio | audio |
audio/x-pn-windows-acm | Windows ACM | audio |
audio/x-pn-windows-pcm | Windows PCM | audio |
audio/basic | 8-bit u-law PCM | audio |
audio/x-pn-au | Sun audio | audio |
audio/3gpp | 3GPP | audio |
audio/3gpp-encrypted | encrypted 3GPP | audio |
audio/scpls | streaming mp3 playlists | audio |
audio/x-scpls | streaming mp3 playlists | audio |
application/smil | SMIL | audio |
application/sdp | Streaming Download Project | audio |
application/x-sdp | Streaming Download Project | audio |
audio/amr | AMR codec | audio |
audio/amr-encrypted | AMR encrypted codec | audio |
audio/amr-wb | AMR-WB codec | audio |
audio/amr-wb-encrypted | AMR-WB encrypted codec | audio |
audio/x-rn-3gpp-amr | 3GPP codec | audio |
audio/x-rn-3gpp-amr-encrypted | 3GPP-AMR encrypted codec | audio |
audio/x-rn-3gpp-amr-wb | 3gpp-AMR-WB codec | audio |
audio/x-rn-3gpp-amr-wb-encrypted | 3gpp-AMR_WB encrypted codec | audio |
application/streamingmedia | Streaming Media | audio |
video/mpeg | MPEG video | video |
audio/x-ms-wma | Windows Media | video |
video/quicktime | QuickTime | video |
video/x-ms-asf | Microsoft ASF | video |
video/x-msvideo | Microsoft AVI | video |
video/x-sgi-mov | SGI movie | video |
video/3gpp | 3GPP video | video |
video/3gpp-encrypted | 3GPP encrypted video | video |
video/3gpp2 | 3GPP2 video | video |
audio/x-realaudio | RealAudio | audio |
text/vnd.rn-realtext | RealText | text |
audio/vnd.rn-realaudio | RealAudio | audio |
audio/x-pn-realaudio | RealAudio plug-in | audio |
image/vnd.rn-realpix | RealPix | image |
application/vnd.rn-realmedia | RealMedia | video |
application/vnd.rn-realmedia-vbr | RealMedia VBR | video |
application/vnd.rn-realmedia-secure | secure RealMedia | video |
application/vnd.rn-realaudio-secure | secure RealAudio | audio |
audio/x-realaudio-secure | secure RealAudio | audio |
video/vnd.rn-realvideo-secure | secure RealVideo | video |
video/vnd.rn-realvideo | RealVideo | video |
application/vnd.rn-realsystem-rmj | RealSystem media | video |
application/vnd.rn-realsystem-rmx | RealSystem secure media | video |
audio/rn-mpeg | MPEG audio | audio |
application/x-shockwave-flash | Macromedia Shockwave | multimedia |
application/x-director | Macromedia Shockwave | multimedia |
application/x-authorware-bin | Macromedia Authorware binary | multimedia |
application/x-authorware-map | Macromedia Authorware shocked file | multimedia |
application/x-authorware-seg | Macromedia Authorware shocked packet | multimedia |
application/futuresplash | Macromedia FutureSplash | multimedia |
application/zip | ZIP | archive |
application/x-lzh | LZH archive | archive |
image/gif | Graphics Interchange Format | image |
image/png | Portable Network Graphics | image |
image/jpeg | JPEG | image |
image/bmp | Microsoft BMP | image |
image/tiff | Tagged Image File Format | image |
image/x-freehand | Macromedia Freehand | image |
image/x-cmu-raster | CMU Raster | image |
image/x-rgb | RGB image | image |
text/css | cascading style sheet | text |
text/html | HTML | text |
text/plain | plain text | text |
text/richtext | rich text | text |
text/tab-separated-values | tab separated values | text |
text/xml | XML | text |
text/xsl | XSL | text |
text/x-sgml | SGML | text |
text/x-vcard | vCard | text |
application/mac-binhex40 | Macintosh BinHex | archive |
application/x-stuffit | Macintosh Stuffit archive | archive |
application/macwriteii | MacWrite Document | document |
application/applefile | Macintosh File | archive |
application/mac-compactpro | Macintosh Compact Pro | archive |
application/x-bzip2 | block compressed | compressed |
application/x-shar | shell archive | archive |
application/x-gtar | gzipped tar archive | archive |
application/x-gzip | gzip compressed | compressed |
application/x-tar | 4.3BSD tar archive | archive |
application/x-ustar | POSIX tar archive | archive |
application/x-cpio | old cpio archive | archive |
application/x-bcpio | POSIX cpio archive | archive |
application/x-sv4crc | System V cpio with CRC | archive |
application/x-compress | UNIX compressed | compressed |
application/x-sv4cpio | System V cpio | archive |
application/x-sh | UNIX shell script | executable |
application/x-csh | UNIX csh script | executable |
application/x-tcl | Tcl script | executable |
application/x-javascript | JavaScript | executable |
application/x-excel | Microsoft Excel | document |
application/mspowerpoint | Microsoft Powerpoint | document |
application/msword | Microsoft Word | document |
application/wordperfect5.1 | Word Perfect | document |
application/rtf | Rich Text Format | document |
application/pdf | Adobe Acrobat | document |
application/postscript | Postscript | documen |
Advanced
- Process HTTPS traffic by SNI (Server Name Indication) if present: If this option is enabled, HTTPS traffic will be categorized using the "Server Name Indication" in the HTTPS data stream, if present. More details in HTTPS Options.
- Process HTTPS traffic by hostname in server certificate when SNI information not present: If this option is enabled and SNI information is not present, the certificate is fetched from the HTTPS server and the server name on the certificate will be used for categorization and filtering purposes.
- Process HTTPS traffic by server IP if both SNI and certificate hostname information are not available: If this option is enabled and neither of the previous options worked, HTTPS traffic will be categorized using the IP address. More details in HTTPS Options.
- Clear Category URL Cache: This option will clear the local cache of categorized sites and URLs. After clearing the cache all new web visits will be looked up fresh using the categorization service. The cache automatically cleans itself as entries become old or stale, so this is mostly for testing.
![](/assets/images/user-manual/um-etm-me/Graphics/Web%20Monitor/apps_web-monitor_advanced.png)
Reports
The Reports tab provides a view of all reports and events for all traffic handled by Web Monitor.
This applications reports can be accessed via the Reports tab at the top or the Reports tab within the settings. All pre-defined reports will be listed along with any custom reports that have been created.
Reports can be searched and further defined using the time selectors and the Conditions window at the bottom of the page. The data used in the report can be obtained on the Current Data window on the right.
Report Entry | Description |
---|---|
Web Monitor Summary | A summary of web monitor actions. |
Web Usage | The amount of total and flagged web requests over time. |
Web Usage (scanned) | The amount of total web requests over time. |
Web Usage (flagged) | The amount of flagged web requests over time. |
Top Categories (by request) | The number of web requests grouped by category. |
Top Categories (by size) | The sum of the size of requested web content grouped by category. |
Top Flagged Categories | The number of flagged web requests grouped by category. |
Top Sites (by request) | The number of web requests grouped by website. |
Top Sites (by size) | The sum of the size of requested web content grouped by website. |
Top Flagged Sites | The number of flagged web requests grouped by website. |
Top Domains (by request) | The number of web requests grouped by domain. |
Top Domains (by size) | The sum of the size of requested web content grouped by domain. |
Top Flagged Domains | The number of flagged web requests grouped by domain. |
Top Domains Usage | The amount of web requests per top domain. |
Top Hostnames (by requests) | The number of web requests grouped by hostname. |
Top Hostnames (by size) | The sum of the size of requested web content grouped by hostname. |
Top Flagged Hostnames | The number of flagged web request grouped by hostname. |
Top Clients (by requests) | The number of web requests grouped by client. |
Top Clients (by size) | The sum of the size of requested web content grouped by client. |
Top Flagged Clients | The number of flagged web request grouped by client. |
Top Usernames (by requests) | The number of web requests grouped by username. |
Top Usernames (by size) | The sum of the size of requested web content grouped by username. |
Top Flagged Usernames | The number of flagged web request grouped by username. |
Top Content (by request) | The number of web requests grouped by category. |
Top Content (by size) | The sum of the size of requested web content grouped by category. |
Top Searches | The number of non blocked, non-flagged search queries grouped by term. |
Top Flagged Searches | The number of flagged search queries grouped by term. |
All Web Events | Shows all scanned web requests. |
Flagged Web Events | Shows all flagged web requests. |
All HTTP Events | Shows all scanned unencrypted HTTP requests. |
All HTTPS Events | Shows all encrypted HTTPS requests. |
All Search Events | Shows all search queries processed by Web Monitor. |
Flagged Search Events | Shows flagged search queries processed by Web Monitor. |
HTTPS Options
There are many ways to handle HTTPS. An overview of the various techniques is described here.
If SSL Inspector is installed and inspects a session, then it is fully decrypted to HTTP before Web Monitor processes the session. In this case HTTPS is treated identically to HTTP. If SSL Inspector is not installed or the session is not inspected, there are still several techniques to handle encrypted HTTP sessions.
- Process HTTPS traffic by SNI (Server Name Indication) if present.
- Process HTTPS traffic by hostname in server certificate when SNI information not present
- Process HTTPS traffic by server IP if both SNI and certificate hostname information are not available.
If Process HTTPS traffic by SNI (Server Name Indication) if present encrypted port-443 traffic will be scanned. Most modern browsers on modern OSs will send the hostname of the server in cleartext - this is called "Server Name Indication" or SNI. SNI is an optional cleartext field in the HTTPS request that shows the hostname of the server. If this option is enabled and the SNI information is present in the HTTPS request, this hostname will be used as the URL for this request and all categorization, flag lists, and pass lists, will be processed as if this were a regular HTTP request to that URL.
If the SNI-based categorization determines the page should be passed (and/or flagged) then the session is allowed and the appropriate event based on the SNI information is logged ("https://example.com/").
For example, if the user visits "https://wellsfargo.com/welcome" in the browser, it will see "wellsfargo.com" as the SNI information. If enabled, the request will be handled exactly like "http://wellsfargo.com" would be. If "Banking" is flagged it will be flagged, unless "wellsfargo.com" is in the pass list or the client IP is in the client IP pass list. If "wellsfargo.com" is flaggeed it will be flagged, unless "wellsfargo.com" is in the pass list or the client IP is in the client IP pass list.
If No SNI information is present and Process HTTPS traffic by hostname in server certificate when SNI information not present is enabled, then the hostname will be pulled from the certificate presented to the client.
For example, if the user visits "https://wellsfargo.com/welcome" in a non-SNI enabled browser, then there is no SNI information. In this case if Process HTTPS traffic by hostname in server certificate when SNI information not present is enabled it will use the certificate information instead to categorize the session. It will download the certificate from the site and see that the certificate is "Issued To" "www.wellsfargo.com." It will use this information to check the category for "https://www.wellsfargo.com" and categorize the session.
If no SNI or certificate information was available and Process HTTPS traffic by server IP if both SNI and certificate hostname information are not available the session will be processed and categorized by IP address. If the IP-based processing and categorization of the web requests determines the session should be flagged, the session is reset and no more processing of this session will be done. If the IP-based processing and categorization determines the page should be passed (and/or flagged) then the session is allowed and the appropriate event based on its IP is logged ("https://1.2.3.4").
For example, if the user visits "https://wellsfargo.com/welcome" in a non-SNI enabled browser, then there is no SNI information. If the the certificate information was missing for some reason then this session can only be identified by IP address. In this case if Process HTTPS traffic by server IP if both SNI and certificate hostname information are not available is enabled it will use the IP address instead. So it will process/categorize this web request as 'http://1.2.3.4' if 1.2.3.4 is the IP of wellsfargo.com. This will still often result in correct categorization for dedicated web servers, but does poorly when using generic cloud computing servers that offer a wide variety of websites.
To see the HTTPS categorization in action use the "All HTTPS Events" query in the event log.
Web Monitor Reports
The Reports tab provides a view of all reports and events for all traffic handled by Web Monitor.
Reports
This applications reports can be accessed via the Reports tab at the top or the Reports tab within the settings. All pre-defined reports will be listed along with any custom reports that have been created.
Reports can be searched and further defined using the time selectors and the Conditions window at the bottom of the page. The data used in the report can be obtained on the Current Data window on the right.
Report Entry | Description |
---|---|
Web Monitor Summary | A summary of web monitor actions. |
Web Usage | The amount of total and flagged web requests over time. |
Web Usage (scanned) | The amount of total web requests over time. |
Web Usage (flagged) | The amount of flagged web requests over time. |
Top Categories (by request) | The number of web requests grouped by category. |
Top Categories (by size) | The sum of the size of requested web content grouped by category. |
Top Flagged Categories | The number of flagged web requests grouped by category. |
Top Sites (by request) | The number of web requests grouped by website. |
Top Sites (by size) | The sum of the size of requested web content grouped by website. |
Top Flagged Sites | The number of flagged web requests grouped by website. |
Top Domains (by request) | The number of web requests grouped by domain. |
Top Domains (by size) | The sum of the size of requested web content grouped by domain. |
Top Flagged Domains | The number of flagged web requests grouped by domain. |
Top Domains Usage | The amount of web requests per top domain. |
Top Hostnames (by requests) | The number of web requests grouped by hostname. |
Top Hostnames (by size) | The sum of the size of requested web content grouped by hostname. |
Top Flagged Hostnames | The number of flagged web request grouped by hostname. |
Top Clients (by requests) | The number of web requests grouped by client. |
Top Clients (by size) | The sum of the size of requested web content grouped by client. |
Top Flagged Clients | The number of flagged web request grouped by client. |
Top Usernames (by requests) | The number of web requests grouped by username. |
Top Usernames (by size) | The sum of the size of requested web content grouped by username. |
Top Flagged Usernames | The number of flagged web request grouped by username. |
Top Content (by request) | The number of web requests grouped by category. |
Top Content (by size) | The sum of the size of requested web content grouped by category. |
Top Searches | The number of non blocked, non-flagged search queries grouped by term. |
Top Flagged Searches | The number of flagged search queries grouped by term. |
All Web Events | Shows all scanned web requests. |
Flagged Web Events | Shows all flagged web requests. |
All HTTP Events | Shows all scanned unencrypted HTTP requests. |
All HTTPS Events | Shows all encrypted HTTPS requests. |
All Search Events | Shows all search queries processed by Web Monitor. |
Flagged Search Events | Shows flagged search queries processed by Web Monitor. |