Class AdaptiveByteStore

  • All Implemented Interfaces:
    ByteStore

    public class AdaptiveByteStore
    extends java.lang.Object
    implements ByteStore
    ByteStore which adopts a hybrid approach between use of memory and use of disk. Bytes are written into an array in memory up to a given size limit; if the amount written exceeds this limit, it's all put in a temporary file instead.

    This class is intended to be a general purpose StoragePolicy implementation that does something sensible most of the time. The details of the implementation may be changed following experience.

    The current implementation uses ByteBuffer.allocateDirect(int) for byte arrays in memory apart from rather small ones. On most OSes this corresponds to using malloc(), thus avoiding heavy use of JVM heap memory. Note very large arrays are still stored on disk, not directly allocated.

    Since:
    5 Nov 2009
    Author:
    Mark Taylor
    • Constructor Summary

      Constructors 
      Constructor Description
      AdaptiveByteStore()
      Constructs a new store with a default memory limit.
      AdaptiveByteStore​(int memLimit)
      Constructs a new store with a given maximum memory limit.
    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()
      Tidies up.
      void copy​(java.io.OutputStream out)
      Takes all the data written so far into this store's sink stream and copies it to a destination stream.
      static int getDefaultLimit()
      Calculates the default memory limit used by this class.
      long getLength()
      Returns the number of bytes currently stored in this object.
      java.io.OutputStream getOutputStream()
      Returns an output stream which can be used to write to the store.
      java.nio.ByteBuffer[] toByteBuffers()
      Returns an array of byte buffers containing the bytes written to this store.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Constructor Detail

      • AdaptiveByteStore

        public AdaptiveByteStore​(int memLimit)
                          throws java.io.IOException
        Constructs a new store with a given maximum memory limit.
        Parameters:
        memLimit - maximum size of in-memory buffer
        Throws:
        java.io.IOException
      • AdaptiveByteStore

        public AdaptiveByteStore()
                          throws java.io.IOException
        Constructs a new store with a default memory limit.
        Throws:
        java.io.IOException
    • Method Detail

      • getOutputStream

        public java.io.OutputStream getOutputStream()
        Description copied from interface: ByteStore
        Returns an output stream which can be used to write to the store. May be called multiple times; always returns the same object. Note that this is not in general buffered - it is the responsibility of the user to take steps like wrapping it in a BufferedOutputStream for efficiency if required.
        Specified by:
        getOutputStream in interface ByteStore
        Returns:
        data sink stream
      • getLength

        public long getLength()
        Description copied from interface: ByteStore
        Returns the number of bytes currently stored in this object.
        Specified by:
        getLength in interface ByteStore
        Returns:
        byte count
      • copy

        public void copy​(java.io.OutputStream out)
                  throws java.io.IOException
        Description copied from interface: ByteStore
        Takes all the data written so far into this store's sink stream and copies it to a destination stream. The output stream is not closed.
        Specified by:
        copy in interface ByteStore
        Parameters:
        out - data destination stream
        Throws:
        java.io.IOException
      • toByteBuffers

        public java.nio.ByteBuffer[] toByteBuffers()
                                            throws java.io.IOException
        Description copied from interface: ByteStore
        Returns an array of byte buffers containing the bytes written to this store. The stored bytes are all of the bytes from the first buffer in the returned array, followed by all in the second, etc. In many cases the returned array will, and probably should, contain a single buffer, but if the written byte count exceeds Integer.MAX_VALUE, more than one will be required. The limit of each buffer indicates the number of bytes it contains.

        Usual usage will be to write all data, then call this method once; this model may affect implementation decisions about efficiency.

        Specified by:
        toByteBuffers in interface ByteStore
        Returns:
        byte buffer containing bytes written
        Throws:
        java.io.IOException - if there is an I/O error
      • close

        public void close()
        Description copied from interface: ByteStore
        Tidies up. Should be called when the data in this object is no longer required. This object may no longer be usable following a call to this method.
        Specified by:
        close in interface ByteStore
      • getDefaultLimit

        public static int getDefaultLimit()
        Calculates the default memory limit used by this class.
        Returns:
        default memory limit