Overview
As a general solution to DNS optimization in the mobile internet era, HTTPDNS mainly addresses the following problems:
Local DNS hijacking/failures
Inaccurate local DNS scheduling
The HTTPDNS SDK for Android mainly provides DNS and cache management capabilities based on HTTPDNS:
When the SDK resolves a domain, it first uses HTTPDNS to get the DNS query result. In extreme cases, when HTTPDNS is unavailable, the result provided by local DNS will be used.
The DNS query result returned by HTTPDNS will carry the TTL information, which the SDK will use to manage the cache storing the result provided by HTTPDNS.
Preparations
2. You have added domains to be resolved in the HTTPDNS console as instructed in Adding a Domain. 4. After activating the service, you have been assigned the configuration information such as authorization ID, AES and DES encryption keys, and HTTPS token. You can also view them on the Development Configuration page.
Configuration information required for using the SDK for Android: Authorization ID: The unique ID of a development configuration used in HTTPDNS, namely, the dnsId
parameter in the SDK used for DNS authentication.
DES encryption key: The dnsKey
parameter in the SDK, which you need to pass in when using the DES encryption method.
AES encryption key: The dnsKey
parameter in the SDK, which you need to pass in when using the AES encryption method.
HTTPS encryption token: The token
parameter in the SDK, which you need to pass in when using the HTTPS encryption method.
SDK Integration
HTTPDNS SDK integration
Importing the AAR package
2. Copy HttpDNSLibs\\HTTPDNS_ANDROID_xxxx.aar
to the corresponding location in the application's libs
folder.
3. Add the following to build.gradle
in the app module:
Note
In v4.3.0, local data storage is added. You also need to add a dependency on Room. For more information, see here. android {
repositories {
flatDir {
dirs 'libs'
}
}
}
dependencies {
implementation(name: 'HTTPDNS_Android_xxxx', ext: 'aar')
implementation "androidx.room:room-rxjava2:2.2.0"
}
Downloading the Maven repository
1. Import dependencies via a POM file.
<dependency>
<groupId>io.github.dnspod</groupId>
<artifactId>httpdns-sdk</artifactId>
<version>4.4.0</version>
<type>aar</type>
</dependency>
<dependency>
<groupId>io.github.dnspod</groupId>
<artifactId>httpdns-sdk</artifactId>
<version>4.4.0-intl</version>
<type>aar</type>
</dependency>
Note
HTTPDNS SDK v4.4.0 supports a version dedicated for Tencent Cloud International. If you want to use the version for Tencent Cloud International, obtain the initialization configuration from the HTTPDNS international console. See Access Documentation for details. Permission configuration
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.READ_PHONE_STATE" />
Network security configuration
If targetSdkVersion of the application is 28 (Android 9.0) or later, HTTP network requests are not allowed by default. For more information, see Opt out of cleartext traffic.
In this case, you need to add the IP used by the HTTPDNS request to the domain allowlist. The configuration is as follows: Configure in the AndroidManifest
file.
<?xml version="1.0" encoding="utf-8"?>
<manifest ... >
<application android:networkSecurityConfig="@xml/network_security_config"
... >
...
</application>
</manifest>
Add the network_security_config.xml
configuration file in the XML directory.
<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
<domain-config cleartextTrafficPermitted="true">
<domain includeSubdomains="false">119.29.29.98</domain>
<domain includeSubdomains="false">119.28.28.98</domain>
</domain-config>
</network-security-config>
SDK initialization
Initialization configuration (supported starting from v4.0.0)
Note
The HTTPDNS SDK provides multiple DNS optimization options, which can be combined for optimal DNS performance.
You can configure setUseExpiredIpEnable(true)
and setCachedIpEnable(true)
to achieve optimistic DNS cache.
The persistent cache feature is used to improve cache hit rate and shorten the time to the first meaningful paint. This feature stores the last DNS result locally and preferentially reads the local cache when an app is launched.
If the cached DNS result has expired (TTL expired), since optimistic DNS cache allows the return of IP addresses in expired DNS results, the SDK returns the IP addresses in the expired DNS result (assuming that the IP addresses are available in most cases) and initiates an async request to update the cache.
When optimistic DNS cache is enabled, cache cannot be hit (there is no cache) for the first DNS request of a domain, so 0;0
is returned, and an async DNS request is initiated to update the cache. Therefore, after enabling optimistic DNS cache, you need to ensure that local DNS will work if no cache is hit. For important domains, we recommend you enable DNS prefetch via preLookupDomains(String... domainList)
.
If the server IP addresses are changed frequently, be sure to enable automatic cache update via persistentCacheDomains(String... domainList)
and DNS prefetch via preLookupDomains(String... domainList)
to ensure the accuracy of DNS results.
Before getting the service instance, you can set some service attributes by initializing the configuration, which can be passed in as configuration items when the SDK is initialized.
Note:
[Disused from v4.4.0] The SDK log reporting capability is configured in the console. For more information, see here. DnsConfig dnsConfigBuilder = DnsConfig.Builder()
.dnsId("xxx")
.dnsKey("xxx")
.dnsIp("xxx")
.desHttp()
.token("xxx")
.logLevel(Log.VERBOSE)
.preLookupDomains("baidu.com", "qq.com")
(Optional) Domains for automatic cache update, in the format of `"baidu.com", "qq.com"`. For the configured domains, the SDK will initiate DNS requests automatically to update their cache when 75% of the TTL has elapsed, so that cache will always be hit for these domains. You can configure up to 10 domains. Configure the domains for automatic cache update and those for DNS prefetch separately.
.persistentCacheDomains("baidu.com", "qq.com")
.ipRankItems(ipRankItemList)
.setCustomNetStack(3)
.setUseExpiredIpEnable(true)
.setCachedIpEnable(true)
.timeoutMills(2000)
.routeIp("XXX")
.enableReport(true)
.build();
MSDKDnsResolver.getInstance().init(this, dnsConfigBuilder);
Initialization for earlier versions (upgrade is recommended)
The service addresses for the HTTP and HTTPS protocols are 119.29.29.98
and 119.29.29.99
respectively (use the 119.29.29.99
IP only when you select an encryption method on your own and channel
is Https
).
The new version of the APIs now can be accessed at 119.29.29.99/98
, and the original HTTPDNS service address 119.29.29.29
is for development and debugging purposes only without an SLA guarantee, so it is not recommended for business purposes. Migrate your business to 119.29.29.99/98
as soon as possible.
When you integrate HTTPDNS through the SDK, if HTTPDNS does not find a DNS query result, the domain will be resolved by local DNS, and the result provided by local DNS will be returned.
SDK integration methods
There are two methods to integrate the DNS capability of the HTTPDNS SDK into the HTTP (HTTPS) network access process:
Method 1: URL replacement
Replace the host part in the URL to get a new URL for accessing the network.
In this method, the URL discards the domain information; therefore, for network requests that require domain information, there is a lot of work to do to ensure compatibility.
Method 2: DNS replacement
Incorporate the DNS capability of HTTPDNS in the network access process to replace local DNS in the original process.
In this method, you don't need to modify the URLs of the requests one by one, so there is no need to conduct extra work to ensure compatibility, but the network library used on the business side must support DNS replacement.
DNS replacement can be implemented by hooking the system's DNS function, but the function is already used in the HTTPDNS SDK, so hooking the function again may cause recursive calls or even stack overflow.
For more information on how to connect to different network libraries, see the corresponding connection documents (in the current directory) and the samples (in the HttpDnsSample
directory).
Compatibility for URL replacement
As described above, for network requests that require domain information (usually when multiple domains are mapped to the same IP), you need to conduct extra work to ensure compatibility. The following describes how to ensure compatibility from the perspective of protocols, and the specific implementation method is subject to the implementation of the network library.
HTTP compatibility
For HTTP requests, you need to notify the server of the domain information by specifying the host field in the header. For more information on the host field, see Host. HTTPS compatibility
HTTPS is a version of HTTP over TLS; therefore, for HTTPS requests, you also need to set the host field.
In HTTPS requests, you should perform a TLS handshake first. During the TLS handshake, the server will send its own digital certificate to you for identity verification; therefore, you also need to notify the server of the domain information during the TLS handshake. In the TLS protocol, you can specify the domain information by using the SNI extension as detailed in Server Name Indication. Using HTTP proxy locally
Note
If an HTTP proxy is used locally, we recommend you not use HTTPDNS to resolve domains.
The following provides a detailed analysis for both integration methods:
URL replacement
According to the HTTP/1.1 protocol, when an HTTP proxy is used, the request line will carry complete server address information on the client. For more information, see origin-form.
In this case (i.e., an HTTP proxy is used locally, the business has been connected to the HTTPDNS SDK by replacing the URL, and the host field has been set correctly), the HTTP request received by the HTTP proxy will contain the server IP information (in the request line) and domain information (in the host field). However, how the HTTP proxy will send the HTTP request to the real destination server depends on the proxy implementation, and the proxy may directly discard the set host field, causing the network request to fail. DNS replacement
Taking the OkHttp network library as an example, after the HTTP proxy is enabled locally, OkHttp will resolve only the host of the configured HTTP proxy but not the host field in the HTTP request. In this case, it is useless to enable HTTPDNS.
Check whether an HTTP proxy is used locally with the following code:
val host = System.getProperty("http.proxyHost")
val port = System.getProperty("http.proxyPort")
if (null != host && null != port) {
}
Integration verification
Log verification
When you pass in true
for the debug
parameter in the init
API to filter logs with the HTTPDNS
tag, if you see logs related to the local DNS (ldns_ip
) and HTTPDNS (hdns_ip
), the connection is correct.
The ldns_ip
key indicates the DNS query results provided by the local DNS.
The hdns_ip
key indicates the DNS query results of A records provided by HTTPDNS.
The hdns_4a_ips
key indicates the DNS query results of AAAA records provided by HTTPDNS.
The a_ips
key indicates the set of IPv4 addresses returned by the DNS API.
The 4a_ips
key indicates the set of IPv6 addresses returned by the DNS API.
Simulating local DNS hijacking
When you simulate hijacking local DNS, if the application works properly, HTTPDNS has been integrated successfully.
Note
Due to the caching mechanism of local DNS, when simulating local DNS for integration verification, make sure that the cache of local DNS has been cleared. You can clear the cache by restarting the server or switching the network. During verification, be sure to compare the effects of local DNS and HTTPDNS.
Modify the hosts file on your server.
Local DNS gets the DNS query result by reading the hosts file on your server first.
You can simulate hijacking local DNS by modifying the hosts file to point the domain to an incorrect IP.
You can directly modify the hosts file on a root server.
Modify the DNS server configuration.
You can simulate hijacking local DNS by modifying the configuration of the DNS server to point it to an unavailable IP (such as another IP on the LAN).
When your server is connected to Wi-Fi, you can modify the DNS server settings by changing the IP settings to Static in the Advanced Settings option of the Wi-Fi (this operation may vary slightly by server).
Use a DNS modifier to modify the DNS server configuration (usually by tampering with the DNS packet through VPN).
Packet capture verification
The following takes accessing a network over HTTP as an example:
Note
Common mobile HTTP/HTTPS packet capture tools (such as Charles and Fiddler) capture packets through an HTTP proxy, so they are not suitable for verifying whether HTTPDNS works. For more information, see Using the HTTP proxy locally. Use tcpdump to capture packets.
On a root server, you can use the tcpdump
command to capture packets.
On a non-root server, the system may have a built-in debugging tool that can be used to get the packet capture result (the enabling method varies by server).
Observe the packet capture result with WireShark.
For HTTP requests, you can observe plaintext information and confirm whether the IP used during request sending is the one returned by the SDK by comparing the log and the specific packet capture record. The details are as shown below:
From the perspective of packet capture, the request for xw.qq.com
is eventually sent to the server at the IP address 183.3.226.35
.
For HTTPS requests, a TLS handshake packet is actually a plaintext packet. After setting the SNI extension (see HTTPS compatibility), you can verify whether the IP used during request sending is the one returned by the SDK by comparing the log and the specific packet capture record. The details are as shown below: From the perspective of packet capture, the request for xw.qq.com
is eventually sent to the server at the IP address 183.3.226.35
.
Notes
getAddrByName
is a time-consuming sync API, so it should be called on a child thread.
If the business on the client is bound to the host, such as the HTTP service of the host or the CDN service, you need to specify the host field of the HTTP header after replacing the domain in the URL with the IP returned by HTTPDNS.
Take URLConnection
as an example:
URL oldUrl = new URL(url);
URLConnection connection = oldUrl.openConnection();
String ips = MSDKDnsResolver.getInstance().getAddrByName(oldUrl.getHost());
String[] ipArr = ips.split(";");
if (2 == ipArr.length && !"0".equals(ipArr[0])) {
String ip = ipArr[0];
String newUrl = url.replaceFirst(oldUrl.getHost(), ip);
connection = (HttpURLConnection) new URL(newUrl).openConnection();
connection.setRequestProperty("Host", oldUrl.getHost());
}
Taking curl as an example, if you want to access www.qq.com
, and the IP obtained by HTTPDNS is 192.168.0.111
, then you can access it as follows:
curl -H "Host:www.qq.com" http://192.168.0.111/aaa.txt
Check whether an HTTP proxy is used locally. If an HTTP proxy is used, we recommend you not use HTTPDNS to resolve domains. Example:
String host = System.getProperty("http.proxyHost");
String port= System.getProperty("http.proxyPort");
if (null != host && null != port) {
}
Was this page helpful?