How to allow Secure FTP protocol (aka FTPS or FTP over SSL/TLS) through ISA server?

This is a little complicated.  I have to copy the whole article from Stefaan Pouseele Blog

When you have to support the Secure FTP protocol (aka FTPS or FTP over SSL/TLS) with ISA Server 2000 you have to take some tough decisions, especially if you have to allow Explicit Security. In that case, the Secure FTP protocol uses the default FTP control connection (TCP port 21). Because the FTP Application Filter can’t and will never understand the Secure FTP protocol, and this is by design, you have to unbind the FTP Application Filter from the FTP protocol in order to support Secure FTP in Explicit Security mode. Of course this breaks the normal FTP support. For more information, check out my FTP article How the FTP protocol Challenges Firewall Security, section ‘5. What about Secure FTP’.

With ISA Server 2004 and 2006 we can solve that Secure FTP dilemma by applying what is explained in the recent ISA Server Product Team Blog article Why do I need a deny rule to make an allow rule for a custom protocol work correctly?. As with HTTP, the binding of the FTP Application Filter to the FTP protocol is a global setting. Moreover, the default FTP protocol definition only specifies the FTP Control connection (primary connection) because the FTP Application Filter handles the Data connections (secondary connections). In other words, unbinding the FTP Application Filter will only allow the FTP Control connection but not the Data connections. So, in order to solve the Secure FTP dilemma we will have to do some more work.

First of all we have to create a custom protocol definition for the Secure FTP protocol (FTPS) as shown in the figure below:

Some important characteristics of the FTPS protocol definition are:

  • The FTPS Control connection (Primary Connections) uses TCP port 21 for the Explicit Security mode and TCP port 990 for the Implicit Security mode.
  • The FTPS Data connection (Secondary Connections) should be defined as Outbound because only FTPS passive mode can work with a NAT relationship. It is also recommended that the Port Range is specified as exactly as possible. If you don’t know on which ports the Secure FTP server will listen for the data connection, you can specify all unprivileged ports > 1023 (1024 – 65534).
  • With the above protocol definition, only Firewall clients will be able to connect to the Secure FTP server due to the secondary connections. If you have to support SecureNAT clients too, you need to adjust the above protocol definition by moving the FTPS Data connection from the Secondary Connections to the Primary Connections section. However, be aware of the security risk associated with specifying such a large port range in the  Primary Connections section.

Next, to allow the FTPS traffic, you need to create two access rules:

  • An access rule that uses the custom FTPS protocol and allows traffic from the source network to the computer objects representing the Secure FTP servers.
  • An access rule that uses the predefined FTP protocol and denies traffic from the source network to the computer objects representing the Secure FTP servers.

Finally, the new allow rule must come before your original rule that allows the normal FTP traffic from the same source network in the ordered list of policy rules, and the new deny rule should be placed immediately after the new allow rule as shown in the figure above.

Here are other references:

https://blogs.technet.com/isablog/archive/2006/09/25/458810.aspx

http://technet.microsoft.com/en-us/library/bb794745.aspx

ISACertTool & ISA Server Enterprise Edition in a Workgroup


Here are some links and ideas, the final solution has yet found.
 
ISA Server 2004 Enterprise Edition in a Workgroup
http://technet.microsoft.com/en-ca/library/cc302483.aspx
Install a computer running ISA Server services (2004 Enterprise Edition)
http://www.microsoft.com/technet/isa/2004/help/SREE_H_InstISASvcs.mspx?mfr=true
Install the ISA Configuration Storage Server and Configure the Firewall Array
http://technet.microsoft.com/en-us/library/cc539144(TechNet.10).aspx
Specify credentials for communication with the Configuration Storage server (2004 Enterprise Edition)
http://www.microsoft.com/technet/isa/2004/help/SREE_H_CSSAuthen.mspx?mfr=true
Specify credentials for communication with the Configuration Storage server (2006 Enterprise Edition)
http://technet.microsoft.com/en-us/library/bb838856(TechNet.10).aspx
ISACertTool for Internet Security and Acceleration (ISA) Server 2006 Enterprise Edition
http://www.microsoft.com/downloads/details.aspx?familyid=655f22ba-2424-4269-94d3-cb07308afc46&displaylang=en
 
ISACertTool for ISA Server 2006 Enterprise Edition
 
DescriptionISA Server 2006 Enterprise Edition uses a Configuration Storage server as a storage mechanism for enterprise and array settings. ISA Server array members must be able to connect to a Configuration Storage server, and certificates are required to authenticate this connection in the following scenarios:
• When ISA Server computers are not installed in a domain (workgroup mode).
• When ISA Server array members are part of a domain that does not have a trust relationship with the domain in which the Configuration Storage server is located.
Certificate configuration is done during ISA Server Setup, but if you want to change configuration settings after installation, ISACertTool.exe helps you do the following:
• Install a server certificate on the Configuration Storage server.
• Install a root certificate on each array member to indicate that it trusts the Certification Authority that issued the server certificate.
To download and install this tool, follow the below steps:
– Click the Download button on this page to start the download.
– Do one of the following:
• To start the installation immediately, click Run.
• To save the download to your computer for installation at a later time, click Save.
• To cancel the installation, click Cancel.

ISA Publishing and Certificate

Some of important info copy-pasted from MS and other sites
 
Very good post on this site recently: http://blogs.technet.com/isablog/default.aspx
 

Install a Server Certificate on the ISA Server Computer

To enable a more secure connection between mobile devices and the ISA Server computer, you must install a server certificate on the ISA server computer. This certificate should be issued by a public Certification Authority because it will be accessed by users on the Internet. If a private Certification Authority is used, the root Certification Authority certificate from the private Certification Authority must be installed on any computer that will need to create a secure (HTTPS) connection to the ISA server computer, as well as the ISA local machine store.

You may perform the following procedures on any server that has IIS installed. Use the following procedures to import a certificate on the ISA server computer.

In this section, you will

Request and install a server certificate from a public Certification Authority

Export the server certificate to a file

Import the server certificate to the ISA server computer

  Note: For a list of public certificate vendors, see Step 6: Certificate Enrollment and Device Provisioning.

Request and Install a Server Certificate from a Public CA

Perform the following procedure to request and install a server certificate on a computer with IIS installed.

To request and install a server certificate from a public CA

1.

In IIS, create a new Web site, pointing the Web site to a new, empty directory.

2.

In IIS Manager, expand the local computer, right-click the Web Sites folder, click New, and then click Web Site to start the Web Site Creation Wizard.

3.

Click Next on the Welcome page.

4.

Type a name for the Web site in the Description field. For example, type ISA Cert Site, and then click Next.

5.

Accept the default settings on the IP Address and Port Settings page.

6.

Enter a path for the Web site on the Web Site Home Directory page. For example, enter c:\temp.

7.

Accept the default settings on the Web Site Access Permissions page and click Next.

8.

Click Finish to complete the Web Site Creation Wizard.

  Important: By default, the new Web site is stopped. You should leave this Web site in the stopped state. There is no reason to start this Web site.
  Note :For more information about creating a new Web site, see IIS product documentation.

9.

Follow the steps provided by the public Certification Authority to create and install a server certificate using the Web site you created in Step 1.

  Important: The important information in the certificate is the common name, or FQDN. Enter the FQDN that will be used by Internet users to connect to the Exchange Outlook Web Access site.
  Note: Confirm that the private key for the certificate that you will install is exportable.

Export the Server Certificate to a File

After the certificate is installed on the Web site that you just created, you will export the certificate to a file. You will then copy this file and import it to the ISA server computer.

Perform the following procedure to export the server certificate that you just installed.

To export the server certificate to a .pfx file

1.

In IIS Manager, expand the local computer, and then expand the Web Sites folder.

2.

Right click the Web site for the Exchange front-end services, by default the Default Web Site, and then click Properties.

3.

On the Directory Security tab, under Secure communications, click Server Certificate to start the Web Server Certificate Wizard.

4.

Click Next on the Welcome page.

5.

Select Export the current certificate to a .pfx file on the Modify the Current Certificate Assignment page.

6.

Type the path and file name on the Export Certificate page. For example, type c:\certificates\mail_isa.pfx, and then click Next.

7.

Enter a password for the .pfx file. This password will be requested when a user is importing the .pfx file. Microsoft recommends that you use a strong password because the .pfx file also has the private key.

  Important: Transfer the .pfx file to the ISA server computer in a secure fashion; it contains the private key for the certificate to be installed on the ISA server computer.

Import the Server Certificate on the ISA Server Computer

Perform the following procedure on the ISA server computer to import the server certificate to the local computer store.

To import a server certificate on the ISA server computer

1.

Copy the .pfx file created in the previous section to the ISA server computer in a secure fashion.

2.

Click Start, and then click Run. In Open, type MMC, and then click OK.

3.

Click File, click Add/Remove Snap-in, and in the Add/Remove Snap-in dialog box, click Add to open the Add Standalone Snap-in dialog box.

4.

Select Certificates, click Add, select Computer account, and then click Next.

5.

Select Local Computer, and then click Finish. In the Add Standalone Snap-in dialog box, click Close, and in the Add/Remove Snap-in dialog box, click OK.

6.

Expand the Certificates node, and right-click the Personal folder.

7.

Select All Tasks, and then click Import. This starts the Certificate Import Wizard.

8.

On the Welcome page, click Next.

9.

On the File to Import page, browse to the file that you previously created and copied to the ISA Server computer, and then click Next.

10.

On the Password page, type the password for this file, and then click Next.

  Note: The Password page provides the option Mark this key as exportable. If you want to prevent the exporting of the key from the ISA server computer, do not select this option.

11.

On the Certificate Store page, verify that Place all certificates in the following store is selected and Certificate Store is set to Place Cert Automatically, and then click Next.

12.

On the wizard completion page, click Finish.

13.

Verify that the server certificate was properly installed. Click Certificates, and then double-click the new server certificate. On the General tab, there should be a note that shows you have a private key that corresponds to this certificate. On the Certification Path tab, you should see a hierarchical relationship between your certificate and the Certification Authority, and a note that shows This certificate is OK.

 

Update Public DNS

Create a new DNS host record in your domain’s public DNS servers. Users will initiate a connection using the name of the Web site. This name must match the common name, or Fully Qualified Domain Name (FQDN), used in the certificate installed on the ISA server computer. For example, a user might browse to https://mail.contoso.com/exchange. In this case, the following conditions must be met for the user to successfully initiate a connection:

The FQDN used in the server certificate installed on the ISA server computer must be mail.contoso.com.

  Important: Contoso.com is a fictitious company domain name used for demonstration purposes in this section, and is not relevant to your specific network. The certificate common name must match the FQDN.

The user needs to resolve mail.contoso.com to an IP address.

The IP address that mail.contoso.com resolves to must be configured on the external network of the ISA server computer.

  Note: For ISA Server Enterprise Edition, if you are working with an NLB-enabled array, the IP address may be a virtual IP address configured for the array. For more information about NLB, see ISA server product Help.

 


 
 

 Request and Configure a Certificate for Your Reverse HTTP Proxy

The root certification authority (CA) certificate for the CA that issued the server certificate on the Web server (the IIS server running your Office Communications Server Web components) needs to be installed on the server running ISA Server 2006. This certificate should match the published FQDN of the external Web farm where you are hosting meeting content and Address Book files.

  • You must install a Web server certificate on your ISA Server. This certificate should match the published FQDN of your external Web farm where you are hosting meeting content and Address Book files.
  • If your internal deployment consists of more than one Standard Edition server or Enterprise pool, you must configure Web publishing rules for each external Web farm FQDN.
 Configure Web Publishing Rules

Use the following procedure to create Web publishing rules.

Note:

This procedure assumes ISA Server 2006 Standard Edition has been installed.

 To create a Web server publishing rule on the ISA Server 2006 computer

  1. Click Start, point to Programs, point to Microsoft ISA Server, and then click ISA Server Management.

  2. In the left pane, expand ServerName, right-click Firewall Policy, point to New, and then click Web Site Publishing Rule.

  3. On the Welcome to the New Web Publishing Rule page, enter a friendly name for the publishing rule, and then click Next. For example, the name of the rule could be OfficeCommunicationsWebDownloadsRule.

  4. On the Select Rule Action page, select Allow, and then click Next.

  5. On the Publishing Type page, select Publish a single Web site or load balancer, and then click Next.

  6. On the Server Connection Security page, select Use SSL to connect to the published Web server or server farm, and click Next.

  7. On the Internal Publishing Details page, enter the FQDN of the internal Web farm that hosts your meeting content and Address Book content in the Internal Site name box, and then click Next. Select from the following options:

    Note:

    The ISA Server must be able to resolve the FQDN to the IP address of the internal Web server. If the ISA Server is not able to resolve the FQDN to the proper IP address, you can select Use a computer name or IP address to connect to the published server, and then in the Computer name or IP address box, enter the IP address of the internal Web server. If you do this, you must ensure that the ISA Server has port 53 opened and can reach an internal DNS server or a DNS server that resides in the perimeter network.
    • If your internal server is a Standard Edition, this FQDN is the Standard Edition server FQDN.
    • If your internal server is an Enterprise pool, this FQDN is the internal Web farm FQDN.
    • On the Internal Publishing Details page, in the Path (optional) box, enter /* as the path of the folder to be published, and then click Next.
      Note:

      In the Web site publishing wizard you can only specify one path. Additional paths can be added by modifying the properties of the rule.
  8. On the Publish Name Details page, confirm that This domain name is selected for Accept Requests for, type the external Web farm FQDN in the Public Name box, and click Next.

  9. On Select Web Listener page, click New to create a new Web listener. This opens the New Web Listener Definition Wizard.

  10. On the Welcome to the New Web Listener Wizard page, type a name for the Web listener in the Web listener name box, and then click Next. For example, type Web Servers.

  11. On the Client Connection Security page, select Require SSL secured connections with clients, and then click Next.

  12. On the Web Listener IP Address page, select External, and then click Select IP Addresses.

  13. On the External Listener IP selection page, select Specified IP address on the ISA Server computer in the selected network, select the appropriate IP address, click Add, and then click OK.

  14. Click Next.

  15. On the Listener SSL Certificates page, select Assign a certificate for each IP address, select the IP address you just added, and then click Select Certificate.

  16. On the Select Certificate page, select the certificate that matches the public name specified in step 9, click Select, and then click Next.

  17. On the Authentication Setting page, select No Authentication, and then click Next.

  18. On the Single Sign On Setting page, click Next.

  19. On the Completing the Web Listener Wizard page, review the Web listener settings, and then click Finish.

  20. Click Next.

  21. On the Authentication Delegation page, select No delegation, but client may authenticate directly, and click Next.

  22. On the User Set page, click Next.

  23. On the Completing the New Web Publishing Rule Wizard page, review the Web publishing rule settings and then click Finish.

  24. Click Apply in the details pane to save the changes and update the configuration.

 To modify the properties of the Web publishing rule

  1. Click Start, point to Programs, point to Microsoft ISA Server, and then click ISA Server Management.

  2. In the left pane, expand ServerName, and then click Firewall Policy.

  3. In the details pane, right-click the secure Web server publishing rule that you created in the previous procedure (for example, OfficeCommunicationsServerExternal Rule), and then click Properties.

  4. On the Properties page, click the From tab:

    • In the This rule applies to traffic from these sources list, click Anywhere, and then click Remove.
    • Click Add.
    • In the Add Network Entities dialog box, expand Networks, click External, click Add, and then click Close.
  5. If you need to publish another path on the Web server, select the Paths tab.

  6. Click Add, type /* for the path to be published, and then click OK.

  7. Click Apply to save changes, and then click OK.

  8. Click the Apply button in the details pane to save the changes and update the configuration.

 Verify or Configure Authentication and Certification on IIS Virtual Directories

Use the following procedure to configure certification on your IIS virtual directories or verify that the certification is configured correctly.

Note:

Perform the following procedure on each IIS Server in your internal Office Communications Server.

The procedure given below is for the Default Web Site in IIS.

 To verify or configure authentication and certification on IIS virtual directories

  1. Click Start, point to All Programs, point to Administrative Tools, and then click Internet Information Services (IIS) Manager.

  2. In Internet Information Services (IIS) Manager, expand ServerName, and then expand Web Sites.

  3. Right-click <default or selected> Web Site, and then click Properties.

  4. On the Web Site tab, ensure that the port number is 443 in the SSL port box, and then click OK.

  5. On the Directory Security tab, click Server Certificate under Secure communications. This opens the Welcome to the Web Server Certificate Wizard.

  6. Click Next.

  7. On the Server Certificate page, click Assign an existing certificate, and then click Next.

  8. On the SSL Port page, ensure that the value is 443 in the SSL port this Web site should use box, and then click Next.

  9. On the Certificate Summary page, verify that settings are correct, and then click Next.

  10. Click Finish.

  11. Click OK to close the Default Web Site Properties dialog box.

 Create a DNS Record

Create an external DNS A record pointing to the external interface of your ISA server, as described in the following section.

 Verify Access through Your Reverse Proxy

Use the following procedure to verify that your users can access information on the reverse proxy. You may need to complete the firewall configuration and DNS configuration before access will work correctly.

 To verify that you can access the Web site through the Internet

  1. Deploy the Live Meeting 2007 client as described in Live Meeting 2007 Client Deployment Guide.

  2. Open a Web browser, type the URLs in the Address bar that are used by clients to access the Address Book files and the Web site for Web conferencing:

    • For Address Book Server, type a URL similar to the following: https://externalwebfarmFQDN/abs/ext where externalwebfarmFQDN is the external FQDN of the Web farm that hosts Address Book server files. User should receive an HTTP challenge, because directory security on the Address Book Server folder is configured to Microsoft Windows® authentication by default.
    • For Web conferencing, type a URL similar to the following: https://externalwebfarmFQDN/conf/ext/Tshoot.html where externalwebfarmFQDN is the external FQDN of the Web farm that hosts meeting content. This URL should display the troubleshooting page for Web conferencing.
    • For distribution group expansion type a URL similar to the following: https://ExternalwebfarmFQDN/GroupExpansion/ext/service.asmx. User should receive an HTTP challenge, because directory security on the distribution group expansion service is configured to Microsoft Windows® authentication by default.
 


 
 
 
ISA Server 2006 Documentation
 
Publishing Exchange Server 2007 with ISA Server 2006
http://go.microsoft.com/fwlink/?LinkID=87060&clcid=0x409
 
Best Practices for Performance in ISA Server 2006
http://go.microsoft.com/fwlink/?LinkID=87155&clcid=0x409
 
ISA Server 2006 Enterprise Edition Installation Guide
http://go.microsoft.com/fwlink/?LinkID=87158&clcid=0x409
 
ISA Server 2006 Standard Edition Installation Guide
http://go.microsoft.com/fwlink/?LinkID=87159&clcid=0x409
 
Authentication in ISA Server 2006
http://go.microsoft.com/fwlink/?LinkID=87068&clcid=0x409
 
Firewall Policy Best Practices for ISA Server 2006
http://go.microsoft.com/fwlink/?LinkID=87160&clcid=0x409
 
Other links:
Other shortcuts and docs:
 

DNS Forwarder vs Root Hints with ISA 2000

Question:
Recently we have experienced a slowdown in resolving names. Our DNS Server uses a forwarder to our ISP. When I connect to the ISP’s modem directly there is no problem resolving addresses (eg nslookup www.microsoft.com) so the problem does not appear to be our ISP’s DNS servers.
When I do the same from either the ISA Server, the DNS server or a client workstation behind the ISA server I cannot resolve. If I remove the forwarder and rely solely on root hints everything works fine.
On the ISA server I have a
1. packet filter for DNS lookup
2. protocol rule with selected protocols DNS Query and DNS Query Server
All of the above worked well until a few weeks ago. There are no events on the DNS Server or ISA Server that seem to relate to the problem.
Why would ISA block a DNS Server from using forwarders but allow resolution of a DNS query via root hints?
Answer:
What DNS server are you using?  Is this a native Windows 2000 or 2003 DNS server?  Is the DNS server inside/behind the ISA server?  And, I assume you are forwarding to the same ISP DNS server that you verified works correctly by querying it directly from outside the ISA server.
Have you tried querying the ISP DNS server directly by using nslookup on various machines behind the ISA server (a client, the DNS server, the ISA server itself)?  Do this with the "server w.z.y.z" command in nslookup, and specify the IP address of the remote server, not it’s name.
Aside from the destination of the lookups, there is very little difference between the DNS queries sent to a forwarder, and those sent to the root servers and other remote nameservers.  The main difference is the "RD" bit (recursion desired).  It is unlikely that ISA would be concerned by that bit.  Still, you can simulate that by sending a non-recursive query through nslookup (set norecurse).  If you do that, and tell nslookup to use ISP DNS server (using the server command), and also enable detailed debugging (set d2), you should see if you get some sort of answers back, and then try a recursive query (set recurse) and see if you get an answer from that.  This might help you diagnose if ISA is interfering with resursive queries.