![]() |
NetCDF
4.5.0
|
The steps for variable name translation are as follows.
Take the set of variables captured above. Thus for the above DDS, the following fields would be collected.
All grid array variables are renamed to be the same as the containing grid and the grid prefix is removed. In the above DDS, this results in the following changes.
It is important to note that this process could produce duplicate variables (i.e. with the same name); in that case they are all assumed to have the same content and the duplicates are ignored. If it turns out that the duplicates have different content, then the translation will not detect this. YOU HAVE BEEN WARNED.
The final netCDF-3 schema (minus attributes) is then as follows.
In actuality, the unlimited dimension is dropped because it is unused.
There are differences with the original libnc-dap here because libnc-dap technically was incorrect. The original would have said this, for example.
Note that this is incorrect because it dimensions S1.FS2.f1(2,2) rather than S1.FS2.f1(2,3).
Any variable (as determined above) that is contained directly or indirectly by a Sequence is subject to revision of its rank using the following rules.
Let the variable be contained in Sequence Q1, where Q1 is the innermost containing sequence. If Q1 is itself contained (directly or indirectly) in a sequence, or Q1 is contained (again directly or indirectly) in a structure that has rank greater than 0, then the variable will have an initial UNLIMITED dimension. Further, all dimensions coming from "above" and including (in the containment sense) the innermost Sequence, Q1, will be removed and replaced by that single UNLIMITED dimension. The size associated with that UNLIMITED is zero, which means that its contents are inaccessible through the netCDF-3 API. Again, this differs from libnc-dap, which leaves out such variables. Again, however, this difference is backward compatible.
If the variable is contained in a single Sequence (i.e. not nested) and all containing structures have rank 0, then the variable will have an initial dimension whose size is the record count for that Sequence. The name of the new dimension will be the name of the Sequence.
Consider this example.
The corresponding netCDF-3 translation is pretty much as follows (the value for dimension Q2 may differ).
Note that for example S1.SQ1.f1_0 is not actually used because it has been folded into the unlimited dimension.
Note that for sequences without a leading unlimited dimension, there is a performance cost because the translation code has to walk the data to determine how many records are associated with the sequence. Since libnc-dap did essentially the same thing, it can be assumed that the cost is not prohibitive.
In an effort to provide better performance for some access patterns, client-side caching of data is available. The default is no caching, but it may be enabled by prefixing the URL with the parameter "cache".
Caching operates basically as follows.
When a URL is first accessed using nc_open(), netCDF automatically does a pre-fetch of selected variables. These include all variables smaller than a specified (and user definable) size. This allows, for example, quick access to coordinate variables. This can be suppressed with the parameter "noprefetch".
Whenever a request is made using some variant of the nc_get_var() API procedures, the complete variable is fetched and stored in the cache as a new cache entry. Subsequence requests for any part of that variable will access the cache entry to obtain the data.
The cache may become too full, either because there are too many entries or because it is taking up too much disk space. In this case cache entries are purged until the cache size limits are reached. The cache purge algorithm is LRU (least recently used) so that variables that are repeatedly referenced will tend to stay in the cache.
The cache is completely purged when nc_close() is invoked.
In order to decide if you should enable caching, you will need to have some understanding of the access patterns of your program.
The ncdump program always dumps one or more whole variables so it turns on caching.
If your program accesses only parts of a number of variables, then caching should probably not be used since fetching whole variables will probably slow down your program for no purpose.
Unfortunately, caching is currently an all or nothing proposition, so for more complex access patterns, the decision to cache or not may not have an obvious answer. Probably a good rule of thumb is to avoid caching initially and later turn it on to see its effect on performance.
Currently, a limited set of client parameters is recognized. Parameters not listed here are ignored, but no error is signalled.
Parameter Name Legal Values Semantics
The OPeNDAP support makes use of the logging facility of the underlying oc system (see http://www.opendap.org/oc). Note that this is currently separate from the existing netCDF logging facility. Turning on this logging can sometimes give important information. Logging can be enabled by using the client parameter "log" or "log=filename", where the first case will send log output to standard error and the second will send log output to the specified file.
Users should also be aware that if one is accessing data over an NFS mount, one may see some .nfsxxxxx files; those can be ignored.
Limited support for configuring the http connection is provided via parameters in the “.dodsrc” configuration file. The relevant .dodsrc file is located by first looking in the current working directory, and if not found, then looking in the directory specified by the “$HOME” environment variable.
Entries in the .dodsrc file are of the form:
That is, it consists of a key name and value pair and optionally preceded by a url enclosed in square brackets.
For given KEY and URL strings, the value chosen is as follows:
If URL is null, then look for the .dodsrc entry that has no url prefix and whose key is same as the KEY for which we are looking.
If the URL is not null, then look for all the .dodsrc entries that have a url, URL1, say, and for which URL1 is a prefix (in the string sense) of URL. For example, if URL = http//x.y/a, then it will match entries of the form
It will not match an entry of the form
because “http://x.y/b” is not a string prefix of “http://x.y/a”. Finally from the set so constructed, choose the entry with the longest url prefix: “http//x.y/a/b]KEY=VALUE” in this case.
Currently, the supported set of keys (with descriptions) are as follows.
HTTP.VERBOSE Type: boolean ("1"/"0") Description: Produce verbose output, especially using SSL. Related CURL Flags: CURLOPT_VERBOSE HTTP.DEFLATE Type: boolean ("1"/"0") Description: Allow use of compression by the server. Related CURL Flags: CURLOPT_ENCODING HTTP.COOKIEJAR Type: String representing file path Description: Specify the name of file into which to store cookies. Defaults to in-memory storage. Related CURL Flags:CURLOPT_COOKIEJAR HTTP.CREDENTIALS.USER Type: String representing user name Description: Specify the user name for Digest and Basic authentication. Related CURL Flags: HTTP.CREDENTIALS.PASSWORD Type: String representing password Type: boolean ("1"/"0") Description: Specify the password for Digest and Basic authentication. Related CURL Flags: HTTP.SSL.CERTIFICATE Type: String representing file path Description: Path to a file containing a PEM cerficate. Related CURL Flags: CURLOPT_CERT HTTP.SSL.KEY Type: String representing file path Description: Same as HTTP.SSL.CERTIFICATE, and should usually have the same value. Related CURL Flags: CURLOPT_SSLKEY HTTP.SSL.KEYPASSWORD Type: String representing password Description: Password for accessing the HTTP.SSL.KEY/HTTP.SSL.CERTIFICATE Related CURL Flags: CURLOPT_KEYPASSWORD HTTP.SSL.CAPATH Type: String representing directory Description: Path to a directory containing trusted certificates for validating server sertificates. Related CURL Flags: CURLOPT_CAPATH HTTP.SSL.VALIDATE Type: boolean ("1"/"0") Description: Cause the client to verify the server's presented certificate. Related CURL Flags: CURLOPT_SSL_VERIFYPEER, CURLOPT_SSL_VERIFYHOST HTTP.TIMEOUT Type: String ("dddddd") Description: Specify the maximum time in seconds that you allow the http transfer operation to take. Related CURL Flags: CURLOPT_TIMEOUT, CURLOPT_NOSIGNAL HTTP.PROXY_SERVER Type: String representing url to access the proxy: (e.g.http://[username:password@]host[:port]) Description: Specify the needed information for accessing a proxy. Related CURL Flags: CURLOPT_PROXY, CURLOPT_PROXYHOST, CURLOPT_PROXYUSERPWD
The related curl flags line indicates the curl flags modified by this key. See the libcurl documentation of the curl_easy_setopt() function for more detail (http://curl.haxx.se/libcurl/c/curl_easy_setopt.html).
For ESG client side key support, the following entries must be specified:
Additionally, for ESG, the HTTP.SSL.CERTIFICATE and HTTP.SSL.KEY entries should have same value, which is the file path for the certificate produced by MyProxyLogon. The HTTP.SSL.CAPATH entry should be the path to the "certificates" directory produced by MyProxyLogon.