CHANGES for Netcdf-Java library
Version 4.0.49.20090825.2329
Build Date = 2009-08-25 23:29:19

Changes in version 4.0

 Release Details:
  49:
   Gempak Station/Surface IOSP improvements
   NetcdfDataset calls CoordSysBuilder.augmentDataset(), then processes scale/missing, then calls buildCoordSys() 
   McIDASSatConvention CoordSysBuilder added
  48.6:
   GempakSoundingFileReader: fix a typo in the vertical coordinate determination code
   AggExistingOne : make time coord into a String
  48.5:
    HDF5: disable region references for now, broken.
    DODNetcdfFile: workaround for Grid HDF4 wierdness (Float64 Longitude[fakeDim2 = 8192])  
    InvDatasetImpl : check if parent is null
    VariableDS only sets orgDataType when its wrapping a variable.
    AggregationOuterDimension tolerates different types between main variable and nested
    AggregationOuterDimension.cacheVar may keep data as different type, make sure to convert to correct type in read()
    MAMath.convert - utility routine for converting Arrays to different types.
    ToolsUI.NcML Tab use XMLEditorKit from bounce library
  48.4:
   AggregationOuterDimension.rebuildDatasets(): bug on updating datasets, assuming agg vars are 1D (!)
   HDF4 images use unshared dimensions
  48.3:
   CFConventions look for standard name attribute for latitude / longitude
  48.2:
   CoordinateAxis2D: do not cache data, messes up strided data access
  48.1:
   udunits numeric timezone bug
  48:
   WRF Conventions looks for GRIDTYPE if it cant find DYN_OPT
   improve M3IO CoordSysBuilder, from Qun He
   Stereographic Projection, from the "natural origin" and the tangent point, calculating the scale factor.
   ToolsUI: Viewer: compare two files, global attributes table
   ToolsUI: NcML Aggregation tab: check for homogeneity.
   AggFmrc: ragged, set missing values to NaN
   DConnect2: allowSessions now defaults to true, so "X-Accept-Session: true" header is sent
     (can use ucar.nc2.dods.DODSNetcdfFile.setAllowSessions() to change)
  47:
   NcMLReader.merge -> mergeNcML : create new Dataset
   setValues.setValues (NcmlReader) : scalar Array need to be reshaped
   new IOSP:  ucar.nc2.iosp.mcidas.McIDASGridServiceProvider
   CF1Convention.getZisPositive() - use in WriterCFPointObsDataset
   atmosphere_hybrid_sigma_pressure_coordinate: ap(z) + b(z)*surfacePressure(x,y)
  46:
   udunit fixes for logarithmic units
   StructurePsuedo: bug in _read(section), returning wrong ArrayStructure.getShape().
   PointTable: refactor and document, esp for CF
   Structure.select(), StructurePseudo() : use member names, not Variable object
   NetcdfWriteable: check for valid DataType, improve error messages. was allowing addVariable(.., DataType.STRING)
     causing invalid vsize. netcdf-java read ok, c library doesnt.
   NcML: logical view (not done)
   NCMLReader: allow use of alternate CrawlableDataset impls in NcML aggregation.
   LambertConformal: crossSeam() true when distance > 20,000.
   ProjectionImpl.latLonToProjBB: dont check crosssSeam()
   ProjectionImpl: now create a new object when needed in latlonToProj(), projToLatlon() - too susceptible to error otherwise.
   NetcdfCFWriter: max 2 GB files.
   NetcdfCFWriter: check if timeAxis is null when dateRange is passed in.
   IOSP and NetcdfFile new methods: .getFileTypeId(), getFileTypeVersion(), getFileTypeDescription()
  45:
   ToolsUI: fine tune point obs display
   ToolsUI: eliminate multiple combobox events by waiting 10 msecs
   dont rely on dim.isUnlimited() to test if theres a record structure
   NetcdfDataset.setDefaultEnhanceMode(Set<Enhance> mode);
   GEMPAK bug workaround (from Don)
   Nexrad2,3 coord caching
   COARDSConvention: withdraw "SimpleUnit.isCompatible("m", unit), must have "positive" attibute
   ncstream uses URLEncode for passing UTF-8 var names.
   N3iosp.sync() checks for numrec < 0 (isStreaming)
   Fmrc: remove global atttributes taken from random prototype dataset
   fix readToByteChannel() problems
   refactor ncstream, another pass at protobuf messsages
   GempakCdm point adapter now does stationAsPoint.
   JoinArray is parameterized
   Table.extraJoins now is a list.
   Stucture.Iterator now handles general rank case. StructureDS.Itertaor is removed
   AvhrrConvention bug
   CoordinateAxis.copyNoCache() must preserve state. caused grid.NetcdfCFWriter to fail when x,y units != km.
  44:
   TDSRadarDatasetCollection improvements
   AvhrrSatellite Convention - start of Image datatype
   ArrayChar.makeString() needs to check string len
   CFpointObs: station unlimited dimension case
   Array.arrayCopy() now deals with src not in canonical order. mostly affected aggregation of datasets which
     do sectioning in memory, such as GINI.
   TDS: remote station data
   add List<Station> StationCollection.getStations(String[] names);
   CdmRemoteDatasetFactory now handles any type of cdmRemote dataset, including Grid.
  43:
   add Variable.setCachedData check for correct array type
   WRF, ADAS CoordSysBuilder makeSoilCoordintae had wrong datatype
   ToolsUI Viewer: write local file
   NetcdfFileWriteable: change att names to legal and give warning rather than exception
  42:
   Fmrc must enhance datasets when acquiring.
   Use FeatureType.ANY instead of FeatureType.NONE
   upgrade protobuf library to 2.1.0
   grib index performance enhancements
   ucar.units enhancement
   nexrad2 bug fixes
   ucar.nc2.dataset.grid is removed, since it can no longer be made to work.
   VerticalTransformCT no longer keeps copy of VerticalTransform, since different coordSys may need different functions.
     Now the VerticalTransform is kept in GridCoordSys, which is the only place that it was used. 
     VerticalTransformCT is now immutable, so it can be shared safely.
   FmrcImpl: dont remove _CoordinateAxes
   ncml: check namespace is correct
   enhanceProxy: throw exception if coordSystem is null
   cdmremote replaces ncstream
   point: add calcBounds() to iterator, collections, datasets
  40:
   CF 1.4 multidim case when time is 1 dimensional
   overflow in IndexChunker when writing large files
   DateType was accepting invalid ISO strings.
   grib indexing: binary indexes dont return unknown params
   grib1 param table reading: make thread safe for TDS
   new visad.jar
   Bufr: was not getting missing data correct when compressed
   add: GridCoordSystem.findXYindexFromLatLonBounded(), findXYindexFromCoordBounded()
  39:
   add uamiv iosp, from Barron Henderson (UNC Chapel Hill)
   HdfEos distinguish between swath and profiles
   GridCoordSys.isGridCoordSys() checks for {x,y} having at least 2 dimensions. to exclude point data
   add CoordinateSystem.makeDomain()
   add NetcdfFile.findGroup()
   Array.getFastIterator is deprecated; remove use in MAMath which fails on section
   add NestedPointFeatureCollection.subset(latLonBB)
   vertical transforms OceanS modified, OceanSG1 and OceanSG2  added, thanks to Sachin Bhate and Rich Signell.
   TdsMonitor application in UI
   IO.readContents(InputStream is, String charset);
   UI: DateFormatMArk in Units panel
   AlbersEqualArea : falseEasting, falseNorthing (with help from R.Schmunk)
   UI: URLDump handles compressed body (shows compressed ratio) and charset.
   NetcdfDataset.sync() now actually works.
   WRF sets MAP_PROJ=0 to flat earth for non-georeferencing runs
   NetcdfDataset.wrap()
   ToolsUI: Fmrc, Aggregation Panels improved
   Fmrc : only use Grids with time coordinate
   NetcdfDataset.enhance() uses Set<Enhance>, not EnumSet<Enhance> in order to get unmodifiable set. Not sure
     if this breaks the API.
   GridDataset always enhances NetcdfDataset with coordSystem, if not already done
   NetcdfDataset.setEnhanceProcessed, isEnhanceProcessed, setCoordSysWereAdded removed
   VariableDS, StuctureDS.clearCoordinateSystems() caleld from NetcdfDataset.clearCoordinateSystems()
   NcML enhance attribute is deprecated - and is now ignored.
  38:
   FmrcImpl - add coordinate systems to component datasets
   CoordinateAxis1DTime.NamedObjecct - do not retain, used only by UI.
   NetcdfFile.finalize() makes TDS run out of memory when heavily loaded
   SimpleUnit gets new UnitFormatManager each time, rather than using a static that becomes threading bottleneck
   GridServiceProvider.readXY() : dont catch Exception
   Fmrc: put "coordinates" attribute back on grid variables.
   new visad.jar (3/20/09)
   GeoGrid.subset() deal with 6 dimensions, 2D time dimension.
   refactor grib iosp: GribGridServiceProvider. binary indexes.
   ToolsUI: added Fmrc/Inventory, NcML/Aggregation Tabs
  37:
   CoordinateTimeAxis1D.findTimeIndexFromDate() now returns 0 when date < first.
   N3iosp.createValidNetcdf3ObjectName() change to new allowable names
   ToolsUI: IOSP/HDF5
   H5header fractal heap.
   Netxrad2, NetcdfFile.makeCompressed - use FileLock to prevent clobbering between threads
   FileCache cleanup scheduling - make sure only one at a time
   make sure NetcdfFile closes raf on Exception in spi.open()
   reinstate NetcdfFile.finalize()
   grib now does filtering on variable names to keep them consistent with nj22.
   joinExisting timeUnitsChange - time variable must be Double, else fractional values get lost.
  36:
   NcMLReader was mistakenly throwing exception when there was no variable shape attribute.
     That is allowed, but deprecated. NcMLWriter now always adds that attribute.
   ucar.units hashCode consistent with equals()
   Gempak IOSP now uses its own Convention, not CF. Add PointFeature processing.
   InvDatasetFmrc prevents FmrcImpl from being closed, with FileCacheNOP
   CoordAxis1D.findIndex() off-by-one bug for irregular descending case.
   FileCache now has name and nicer log messages; log to 'cacheLogger'
   NetcdfFile.isClosed didnt work for cache
  35:
    NcML: <aggregation> allows elements to modify each dataset in the aggregation, before it is aggregated.
      this allows datasets to be made into grids for fmrc, for example.
    Some improvements in NcML error messages.
    CFpointObs improved, handles gempak structures.
    JoinArray added, Join interface clarified
    Table.getStructureDataIterator(Cursor cursor, ...)
    add _CoordinateSystemFor to allow wholesale assignment of coord systems to variables.
    Gempak IOSP now standard
    cleanup NetcdfFile.isClosed(). Make sure you get error if you call read() after close, even in cache/
  34:
    Fmrc.makeDatasets must clear coordinate system.
    GINI polar stereographic bug
    grid/grib/gempak refactor
    ToolsUI/FeatureTypes/WMS is rudimentory WMS client
    FeatureDataset extends FileCacheable, so they can be cached directly
  33: refactor ThreddsDataFactory
    add Radial datasets to FeatureDatasetFactoryManager; TypedDatasetFactory now obsolete
    experimental PointDatasetFromNcStream
    fysat iosp
    grib refactor
  32: Stereographic false_east/north implemented
    CoordinateAxis.section() uses copy() not copying all fields.
    Move CoordinateAxis.POSITIVE_XX to ucar.nc2.constants.CF
    GDV Conventions: dont check for meters but COARDS has to
    CFPointObs station ragged 
    put thredds/wcs back into netdf.jar for now
    NetcdfDataset.main() now is a cover to FileWriter.
    CoordinateAxis1D - dont convert misssing values to NaNs, missing values not alllowed so likely a user metadata error
    ArrayChar.makeFromString()
    CFPointObWriter more bugs
    N3streamWriter takes an optional numrec, if numrec are know in advance to avoid new streaming format
    NetcdfDataset : disambiguate bare http URL requests, use HttpClient
  31: UnidataPointObs : allow POINT type even if really a STATION.
    NSSLRadarConvention checks no Convention is set.
    COARDS : look for units of meters for vertical coords (again), when all else fails.
    DefaultCoordBuilder no longer extends COARDS, to avoid such nonsense.
    ncStream uses protocol buffers (experimental)
    nimbus NPE
    BufrTable UI: dump  distinct DDS
    RAF: fix writeToStream
    Array.factory(ByteBuffer)
    bug in PointFeatureSubset iterator.
    move Station, EarthLocation to ucar.unidata.geoloc
  30: Modis Satellite CoordSysBuilder
    Cosmic profiles
    HDF5 fletcher32
    jpeg2000Unpacking Exception
    Mercator false east/northing
    Nimbus/RAF Convention: try to guess more coordinates
    add ucar.ma2.StructuresDataComposite
    fix CFPointObWriter to correctly handle CHAR
    add FeatureType selection to PointFeature tab
    context menu on FeatureScan
    FeatureDatasetFactory was not threadsafe, isMine() returns "analysis object"
    PointFeatures: standard package refactor; add Table.Type.Top for top-level variables
    PointFeatureDatasetViewer: works with all types except Section. add 100 obs limit for speed
    Bug in JoinExisting aggs with > 1 dataset, when taking a section of a cached variable.
    Try to reduce memory usage in CrawlableDataset.
    Grib iosp: dont swallow exception on data reading error
    NetcdfFile : remove finalize() for now
    N3iosp.close() : check for null raf, set raf to null on close.
    remove VariableDS.isCoordinateVariable() - must use classic sense of coordinate variable
    add Aggegation info to NetcdfDataset.getDetailInfo()
    refactor NcML Aggregation
    MetadataExtractor.extractDateRange() - deal with no time axis at all.
  29: Rotated Lat/lon projection
    JoinExisting caching now handles nested aggregations, by using the id attribute to identify them.
    GridDataset, RadialDataset: use Formatter instead of StringBuilder
    VariableIF use Formatter instead of StringBuilder
    CoordTransBuilder use Formatter instead of StringBuilder
    add getCatalogRef(InvCatalogRef dd) to CatalogGrawler.Listener
    CoordAxis1D.findCoordElement() bug for Longitude
    MAMath.convert2packed() utility for scale/offset writing
    Change _unsigned to _Unsigned; look for _Unsigned but ignore case.
  28: EarthLocation.isMissing()
    Improvements to CoordAxis1D.findCoordElement(coord)
    Start to fix CoordSysBuilder to deal with groups.
    DefaultConventions - split from GDV.
    Nested Tables implementation FeatureType (ucar.nc2.ft) framework. mostly POINT, some STATION
    Bugs and performance improvements for StructureDS
    TDWR into NIDS iosp
    Grib2 tolerate missing parameters
  27: Start implementing CF point data spec in FeatureType (ucar.nc2.ft) framework
    Point data: RecordHelper using wrong StructureMembers - always need to get from curent StructureData/Array object!
    Range, Section: use Range.VLEN for size < 0.
    H5iosp: structure member reading had wrong byteOrder
    VariableDS.lookupEnum must defer to orgVar
    H5header: redo String, char attributes;  _FillValue attribute vs Message
    H5iosp: exception if unknown filter.
    HDF5 shuffle filter was inverted
    HDF5 char attributes are Strings (UTF-8)
    DataType.getClass is deprecated - use DataType.getPrimitiveClass
    Array.isUnsigned(): ArrayByte, ArrayShort, ArrayInt all check when doing conversions;
      NetcdfFile.readData() sets this based on Variable.isUnsigned().
    DataType.OPAQUE : use ByteBuffers wrapped by ArrayObject
    Array.set() made public for efficiency, eg in Attributes
    bugs in enhance handling and NcML
    JNI interface to netcdf4 C library, for testing
  26: ucar.nc2.ft.point.writer.CFPointObWriter, WriterCFPointObsDataset for writig point obs data in proposed CF format.
    EnhancedScaleMissingImpl: deal with unsigned data and missing values
    Universal Format Radar IOSP
    NetcdfDatasetInfo must reopen dataset, capture CoordSysBuilder info, so that normal case doesnt retain unneeded objects
  25: opendap parsing bug with signed bytes
    ncml agg fixes
    grib thread-safety
    Calendar.clear() to eliminate random millisecs in a new Date
    NIDS: raster,  DPA read bug
    BUFR tables cleanup
    add Grib GDS type 87
    NCDumpW -ncml -vall now works
    cache cleanup only worked once (!)
    agg files were being opened twice under some circumstances
    FileCache hard limit
  24: cleanup error messages when catref or show catalog service fails to open remote catalog.
    HTTPRandomAccessFile tolerates lack of "Accept-Ranges" header
    bug in IO.copyUrlB buffering - was very slow, prone to hang
    start to reduce memory consumption based on TDS profiling:
        - remove ucar.ma2.Index.set/getName() since its never used
        - NetcdfDataset does not retain NetcdfDatasetInfo
        - remove GribServiceProvider.parseInfo
        - DateUnit does not retain DateFormatter or timeUnitString
        - LatLonPointImpl does not preallocate StringBuilders
  23: new visad.jar (9/12/08)
    more BUFR fixes
    DateRange now moves when start or end date = "present"
    NetcdfFile.canOpen()
    GeoCoordSys: warn, dont fail, if x, y axis units not compatible with "km"
    mcidas area iosp
  22: improved BUFR reading
  21: Improve nexrad3 iosp
    GeoTiff: new projections
  20: StructureDS: get nested convert to work
    Mercator projection: remove lat0; grib parameters were interpreted wrong
    Allow opening GRIB files where we dont know the grid type, and so cant add coordinates
    HDF5 bug: filtered data (eg compressed) wasnt checking for byte order.
    add 'nodods':, 'slurp:' as aliases for http:
    enums return values, enhance will convert to Strings
    redo NetcdfDataset.EnhanceMode, use EnumSet
    ENUM1, ENUM2, ENUM4 enum types
    BUFR code tables (still experimental)
    BUFR "associated variables"
    VariableDS extends EnhanceScaleMissing, StructureDS and VariableEnhanced does not
  19: BUFR reading mostly works
    turn off debugging
    StructureDS.convert nested structures
  18: fix large file writing
    in agg, fix CacheVar, joinExisting persistance.
    move NetcdfDataset.makeArray to Array.makeArray, add new variant
    UnidataPointObsDataset does not require bounding box in global attributes
    NetcdfFileWriteable will allow object names to start with a digit
    ToolsUI : allow setting file caching
    add NcdumpW.printArray for vanilla array printing
    CoordSysBuilder allows coordinate axes to have coordinate systems
    move McIDASAreaProjection, McIDASAreaTransformBuilder to ucar.nc2.iosp.mcidas
    change AxisType, DataType, FeatureType, TransformType into a java 1.5 enum
    WRFConvention tolerates not having GLAT,GLON fields
    bug fixes in BUFR
    NcMLWriter.writeXMLexplicit puts <explicit/> in NcML
    fix bug in NCdumpW.print sequences
  17: HTTPRandomAccessFile allows content-length to be long
    fix NcML aggregation unit tests; add new ones
    integrate aggregation with FileCache, experimental concurrency
    fix bug with relative URLs with blanks in them (URLnaming)
    finish new agg features : promoteGlobalAttribute, timeUnitsChange
  16: remove direct use of ucar.units, except from ucar.nc2.units
    rewrite NetcdfFileCache, remove NetcdfDatasetCache
    EnhanceScaleMissingImpl uses Misc.closeEnough for missing value comparisions.
    default VariableDS.useNaNs = true
    ucar.nc2.util.cache.FileCache uses FileCacheable, FileFactory interfaces;
        implement NetcdfFileCache, HTTP file cache with FileCache.
    start adding net.jcip.annotations to document thread safetyAgg
    convertScaleOffsetMissing() methods back into VariableEnhanced
    clean up Aggregation, make sure docs are correct
    clean up NetcdfDataset.EnhanceMode, add deferred option
  15: change StringBuffer to StringBuilder for efficiency
    clean up ArrayStructure/StructureData API, clarify Member use
    StructureData.getScalarXXX(name) must be correct type
    cleanup javadoc, improve reference docs
    NetcdfFileWriteable - redefine mode, large file support, extra header bytes
    library upgrades : JDOM 1.1
    NetcdfFile.readSection(secSpec) used in NcDumpPanel, replaces old readNestedData.
  14: NetcdfDataset: unsigned always gets converted
    Netcdf-3 files: truncate Strings at first null byte.
    complete rework of point datatypes, now in package ucar.nc2.ft (feature types)
    many improvements to Structure, ArrayStructure etc
    rewrite BUFR and supporting library, improve tables, etc
    make sure compiles under 1.5
  13: hdf4 attribute arrays
  12: DODS preload
    put back HttpClientManager, RuntimeConfigParser temporarily (testing with IDV)
    hdf4 bug fixes
    experiment with ucar.nc2.dt2
    iosp.NmcObsLegacy
    DataType Sequence
  11: start to fix name/shortName, ie deal with Groups systematically
    Structure/ArrayStructure/StructureData refactor
  10: hdf4 bug fix - no data tag, use missing data
    Structure.subset(), nc3 optimizes for it
    hdf5, ArrayStructureBB : set byteOrder
    ArrayStructureBB puts Strings on "heap"
    rearrange packages to separate modules
    cdm build redone - xxxComplete.jar added
    fix problem preventing 1.5 compilation (ternary op returns array)
  09: hdf5 byteOrder must be set when chunked (tiledLayout)

  ---------------------

  Overview of changes from 2.2

   Data Model
    * Object names are Unicode Strings. No Restrictions on characters used.
    * DataType STRING are Unicode. In Netcdf3/4, stored as UTF-8 encoded.
    * DataType CHAR are uninterpreted bytes, default is to consider them ascii.
    * DataType ENUM : stored as 1,2,4 byte ints, map to Strings
    * DataType OPAQUE : byte[] cant be subsetted.
    * DataType SEQUENCE : Structure(*) only accessible through StructureDataIterator

   Features:
    netcdf
      * Structure.select(): create subset consisting only of selected member variables
      * StructureDS correctly gets enhanced
      * NetcdfFileWriteable supports large file writing, redefine mode, extra header bytes, preallocating file size,
         all for Netcdf-3 files.
    iosp
     * netcdf-4
     * complete HDF5 access
     * complete HDF4 access
     * parsing of HDF-EOS/HDF5-EOS metadata to identify dimensions.
     * GEMPAK grids
     * NMC Office Note 29/143
     * BUFR (edition 2-4)
     * Mcidas AREA
     * Universal Radar Format
    geoloc
     * Add RotatedLatLon projection
     * Add RotatedPole projection
    ft
     * Point Feature Types completely reworked, based on StructureData iterators
    NcML
     * Tiled aggregation
     * A JoinExisting type now accepts timeUnitsChange, ndicating time units change in the differnt files
     * A JoinExisting type now accepts dateFormatMark, if theres only one time step per file, to specify the agg coord value
     * Union accepts scan
     * Global attributes can be promoted to Variables.
     * Can specify variables to cache.
     * Can specify iosp, a string parameter to the iosp, and the RAF buffer size

   Libraries
    * JDOM 1.1
    * updated visad 9/12/08
