AUTOMATIC RULE COACHING

A method of validating rules configured to be utilized in an information extraction application, including: receiving a plurality of labeled samples in a training database; for each of the rules in the rule database: (a) determining, for each of the data points of the plurality of labeled samples in the training database to which the rule applies, whether applying the rule to the data point has a positive or negative impact on matching an output for the data point based on the rule to the assured output of the labeled sample corresponding to the data point; (b) generating positive impact information for the rule based on the positive voters; (c) generating negative impact information for the rule based on the negative voters; and (d) determining a metric for the rule based on the quantity of the negative voters and the quantity of the positive voters; ranking the rules based on the metrics corresponding to the rules; and sending to a user for refinement one or more flagged rules of the rules that have a lowest ranking of the metric. Other embodiments are provided.