Class ClassifierSubsetEval

  • All Implemented Interfaces:
    java.io.Serializable, ErrorBasedMeritEvaluator, SubsetEvaluator, CapabilitiesHandler, OptionHandler, RevisionHandler

    public class ClassifierSubsetEval
    extends HoldOutSubsetEvaluator
    implements OptionHandler, ErrorBasedMeritEvaluator
    Classifier subset evaluator:

    Evaluates attribute subsets on training data or a seperate hold out testing set. Uses a classifier to estimate the 'merit' of a set of attributes.

    Valid options are:

     -B <classifier>
      class name of the classifier to use for accuracy estimation.
      Place any classifier options LAST on the command line
      following a "--". eg.:
       -B weka.classifiers.bayes.NaiveBayes ... -- -K
      (default: weka.classifiers.rules.ZeroR)
     -T
      Use the training data to estimate accuracy.
     -H <filename>
      Name of the hold out/test set to 
      estimate accuracy on.
     
     Options specific to scheme weka.classifiers.rules.ZeroR:
     
     -D
      If set, classifier is run in debug mode and
      may output additional info to the console
    Version:
    $Revision: 5511 $
    Author:
    Mark Hall (mhall@cs.waikato.ac.nz)
    See Also:
    Serialized Form
    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void buildEvaluator​(Instances data)
      Generates a attribute evaluator.
      java.lang.String classifierTipText()
      Returns the tip text for this property
      double evaluateSubset​(java.util.BitSet subset)
      Evaluates a subset of attributes
      double evaluateSubset​(java.util.BitSet subset, Instance holdOut, boolean retrain)
      Evaluates a subset of attributes with respect to a single instance.
      double evaluateSubset​(java.util.BitSet subset, Instances holdOut)
      Evaluates a subset of attributes with respect to a set of instances.
      Capabilities getCapabilities()
      Returns the capabilities of this evaluator.
      Classifier getClassifier()
      Get the classifier used as the base learner.
      java.io.File getHoldOutFile()
      Gets the file that holds hold out/test instances.
      java.lang.String[] getOptions()
      Gets the current settings of ClassifierSubsetEval
      java.lang.String getRevision()
      Returns the revision string.
      boolean getUseTraining()
      Get if training data is to be used instead of hold out/test data
      java.lang.String globalInfo()
      Returns a string describing this attribute evaluator
      java.lang.String holdOutFileTipText()
      Returns the tip text for this property
      java.util.Enumeration listOptions()
      Returns an enumeration describing the available options.
      static void main​(java.lang.String[] args)
      Main method for testing this class.
      void setClassifier​(Classifier newClassifier)
      Set the classifier to use for accuracy estimation
      void setHoldOutFile​(java.io.File h)
      Set the file that contains hold out/test instances
      void setOptions​(java.lang.String[] options)
      Parses a given list of options.
      void setUseTraining​(boolean t)
      Set if training data is to be used instead of hold out/test data
      java.lang.String toString()
      Returns a string describing classifierSubsetEval
      java.lang.String useTrainingTipText()
      Returns the tip text for this property
      • Methods inherited from class java.lang.Object

        equals, getClass, hashCode, notify, notifyAll, wait, wait, wait
    • Constructor Detail

      • ClassifierSubsetEval

        public ClassifierSubsetEval()
    • Method Detail

      • globalInfo

        public java.lang.String globalInfo()
        Returns a string describing this attribute evaluator
        Returns:
        a description of the evaluator suitable for displaying in the explorer/experimenter gui
      • listOptions

        public java.util.Enumeration listOptions()
        Returns an enumeration describing the available options.
        Specified by:
        listOptions in interface OptionHandler
        Returns:
        an enumeration of all the available options.
      • setOptions

        public void setOptions​(java.lang.String[] options)
                        throws java.lang.Exception
        Parses a given list of options.

        Valid options are:

         -B <classifier>
          class name of the classifier to use for accuracy estimation.
          Place any classifier options LAST on the command line
          following a "--". eg.:
           -B weka.classifiers.bayes.NaiveBayes ... -- -K
          (default: weka.classifiers.rules.ZeroR)
         -T
          Use the training data to estimate accuracy.
         -H <filename>
          Name of the hold out/test set to 
          estimate accuracy on.
         
         Options specific to scheme weka.classifiers.rules.ZeroR:
         
         -D
          If set, classifier is run in debug mode and
          may output additional info to the console
        Specified by:
        setOptions in interface OptionHandler
        Parameters:
        options - the list of options as an array of strings
        Throws:
        java.lang.Exception - if an option is not supported
      • classifierTipText

        public java.lang.String classifierTipText()
        Returns the tip text for this property
        Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui
      • setClassifier

        public void setClassifier​(Classifier newClassifier)
        Set the classifier to use for accuracy estimation
        Parameters:
        newClassifier - the Classifier to use.
      • getClassifier

        public Classifier getClassifier()
        Get the classifier used as the base learner.
        Returns:
        the classifier used as the classifier
      • holdOutFileTipText

        public java.lang.String holdOutFileTipText()
        Returns the tip text for this property
        Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui
      • getHoldOutFile

        public java.io.File getHoldOutFile()
        Gets the file that holds hold out/test instances.
        Returns:
        File that contains hold out instances
      • setHoldOutFile

        public void setHoldOutFile​(java.io.File h)
        Set the file that contains hold out/test instances
        Parameters:
        h - the hold out file
      • useTrainingTipText

        public java.lang.String useTrainingTipText()
        Returns the tip text for this property
        Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui
      • getUseTraining

        public boolean getUseTraining()
        Get if training data is to be used instead of hold out/test data
        Returns:
        true if training data is to be used instead of hold out data
      • setUseTraining

        public void setUseTraining​(boolean t)
        Set if training data is to be used instead of hold out/test data
        Parameters:
        t - true if training data is to be used instead of hold out data
      • getOptions

        public java.lang.String[] getOptions()
        Gets the current settings of ClassifierSubsetEval
        Specified by:
        getOptions in interface OptionHandler
        Returns:
        an array of strings suitable for passing to setOptions()
      • buildEvaluator

        public void buildEvaluator​(Instances data)
                            throws java.lang.Exception
        Generates a attribute evaluator. Has to initialize all fields of the evaluator that are not being set via options.
        Specified by:
        buildEvaluator in class ASEvaluation
        Parameters:
        data - set of instances serving as training data
        Throws:
        java.lang.Exception - if the evaluator has not been generated successfully
      • evaluateSubset

        public double evaluateSubset​(java.util.BitSet subset)
                              throws java.lang.Exception
        Evaluates a subset of attributes
        Specified by:
        evaluateSubset in interface SubsetEvaluator
        Parameters:
        subset - a bitset representing the attribute subset to be evaluated
        Returns:
        the error rate
        Throws:
        java.lang.Exception - if the subset could not be evaluated
      • evaluateSubset

        public double evaluateSubset​(java.util.BitSet subset,
                                     Instances holdOut)
                              throws java.lang.Exception
        Evaluates a subset of attributes with respect to a set of instances. Calling this function overides any test/hold out instancs set from setHoldOutFile.
        Specified by:
        evaluateSubset in class HoldOutSubsetEvaluator
        Parameters:
        subset - a bitset representing the attribute subset to be evaluated
        holdOut - a set of instances (possibly seperate and distinct from those use to build/train the evaluator) with which to evaluate the merit of the subset
        Returns:
        the "merit" of the subset on the holdOut data
        Throws:
        java.lang.Exception - if the subset cannot be evaluated
      • evaluateSubset

        public double evaluateSubset​(java.util.BitSet subset,
                                     Instance holdOut,
                                     boolean retrain)
                              throws java.lang.Exception
        Evaluates a subset of attributes with respect to a single instance. Calling this function overides any hold out/test instances set through setHoldOutFile.
        Specified by:
        evaluateSubset in class HoldOutSubsetEvaluator
        Parameters:
        subset - a bitset representing the attribute subset to be evaluated
        holdOut - a single instance (possibly not one of those used to build/train the evaluator) with which to evaluate the merit of the subset
        retrain - true if the classifier should be retrained with respect to the new subset before testing on the holdOut instance.
        Returns:
        the "merit" of the subset on the holdOut instance
        Throws:
        java.lang.Exception - if the subset cannot be evaluated
      • toString

        public java.lang.String toString()
        Returns a string describing classifierSubsetEval
        Overrides:
        toString in class java.lang.Object
        Returns:
        the description as a string
      • main

        public static void main​(java.lang.String[] args)
        Main method for testing this class.
        Parameters:
        args - the options