Overview:
This error is seen when Enterprise Vault is installed in the non default instance of SQL.  In order for the Deployment Scanner to communicate successfully, the SQL Server named instance must be used instead of the fully qualified server name of the SQL Server.  When running the Deployment Scanner Server Configuration the Microsoft SQL Server named instance should be entered.

Example:
SQLServer.MyDomain.com\InstanceName


Solution:
Enter the instance name in the format above in the Microsoft SQL Server section in the Server Configuration of the Deployment Scanner.  (For more information on the Deployment Scanner, see article 289859 in the Related Documents section.)

[For Windows Server 2008]
If you run deployment scanner on Windows Server 2008 environment (x64 as well) , need to check your FIREWALL setting carefully. (e.g. temporary disable the firewall and try its connection)
May requires to run SQL Server Browser in SQL Server as well.