Home

Jaxon provides a multi-user labeling capability that allows users to add new labels to and vote on existing labels for datasets with varying degrees of supervision.

On this page you will find:

• The Basics of How to Start Labeling
• Deep Dive - Labeling
• Deep Dive - Voting
• Labeling Modes Explainer

The Basics of How to Start Labeling

1. Choose between Manual and Guided Modes by clicking on the Manual button

1. Select a dataset to label from the dropdown. All datasets within the selected project will be available.

1. If you are in Manual mode, select the labeling Mode. If you are in Guided mode, this will be greyed out.

<aside> <img src="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/2d14f4e0-748c-461e-b82f-0fc4ec99858c/0_JAXON_Logo_Mark_2_2.jpg" alt="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/2d14f4e0-748c-461e-b82f-0fc4ec99858c/0_JAXON_Logo_Mark_2_2.jpg" width="40px" /> More information on labeling modes below.

</aside>

1. After a dataset has been selected, the Labels tab will have a similar layout to the following for labeling

1. After a dataset has been selected, the Labels tab will have a similar layout to the following for voting.

<aside> <img src="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/f0334d97-8fbc-4c2a-ae2a-12c1649a50df/0_JAXON_Logo_Mark_2_2.jpg" alt="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/f0334d97-8fbc-4c2a-ae2a-12c1649a50df/0_JAXON_Logo_Mark_2_2.jpg" width="40px" /> For more details on how to label and/or vote, see below.

</aside>

Back to the top ↑

Deep Dive - Labeling

1. If you are Labeling, the current example only will display towards the center of the screen

1. Classes can be assigned by the user to the right of the displayed example

1. The classes displayed within the labeling box correspond to the specification assigned to the current project.

1. Labels can be typed into the text box by the user. Select the correct label from the dropdown that appears or press the return key

1. Labels can also be selected by clicking on the names of the classes within the labeling box.

1. Users can assign a confidence level to the label with the number of stars to indicate to Jaxon how certain they are about how to label the example. The default is 2 stars.

1. When a label (or several labels in a multi-label scenario) is successfully selected, it will display above the example

1. The label can be removed by clicking on the red X displayed next to the label

1. Users may request that Jaxon suggest a label for the example by pressing the Suggestion button.

1. If none of the labels match, the No Match button can be selected to assign a label of “none” to the example.

1. Once you are satisfied with the label assigned, the labeled example can be submitted by pressing the return key or the Submit button.

1. The number of examples that have been labeled is displayed here. Go back to old examples or skip ahead using these arrows.

<aside> <img src="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/3f6a785c-604b-4cf8-83e7-7aee38f5aaed/0_JAXON_Logo_Mark_2_2.jpg" alt="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/3f6a785c-604b-4cf8-83e7-7aee38f5aaed/0_JAXON_Logo_Mark_2_2.jpg" width="40px" /> A label must be submitted in order to be assigned to the example.

</aside>

Back to the top ↑

Deep Dive - Voting

Voting allows the user to vote on the accuracy of an example that has already been assigned a label. Voting helps reinforce confidence scores for models and ensembles.

1. If you are Voting, the current example only will display towards the center of the screen. A label will already be assigned to the example, shown in the grey bar.

1. Only the bottom portion of the voting box will be used

1. The classes displayed above the voting box correspond to the specification assigned to the current project.

1. Users can assign a confidence level to their vote with the number of stars to indicate to Jaxon how certain they are about their vote on the example’s label. The default is 2 stars.

1. The number of examples that have been voted on is displayed here. Go back to old examples or skip ahead using these arrows.

1. If you agree with the assigned label, select Yes to move on to the next example

1. If you disagree with the assigned label, select No to move on to the next example

Back to the top ↑

Labeling Modes Explainer

The Jaxon platform contains several labeling modes that allow the user to apply labels and votes to their dataset with varying degrees of supervision.

<aside> <img src="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/eef356ce-0875-4701-a71e-a4c6b9ae5964/0_JAXON_Logo_Mark_2_2.jpg" alt="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/eef356ce-0875-4701-a71e-a4c6b9ae5964/0_JAXON_Logo_Mark_2_2.jpg" width="40px" /> If you’re starting from a completely unlabeled dataset, we suggest you first apply a nominal number of labels to the dataset by using the Random Labeling option with Manual Mode. Once there are 5-7 examples of each class represented in the specification, Guided Mode and the other labeling and voting modes will be much more effective.

</aside>

• Manual Mode - allows the user to select which labeling or voting mode they wish to label the dataset with.
• Guided Mode - in Guided Mode, Jaxon automatically selects the optimal mode for a batch of examples, aiming to deliver a balanced distribution while containing the cost of labeling.
• Random Labeling - random examples are served up for labeling**.**
• Active Labeling - an example is picked for the user based on the active learning technique. This mode requires a sufficient number of labels to be present to bootstrap a classifier (5-7 labels per class). As more labels are applied, the classifier is updated.
• Outlier Detection - Outlier detection can be used to identify those examples that are the most different from the rest. While this can surface model errors, it can also surface the odd “corner cases” that would otherwise be lost to sampling from the “common case” examples.
• Voting - labeled examples are randomly served for label confirmation.
• Active Voting - this mode picks an unlabeled example and suggests a label that users vote either ‘Yes’ or ‘No’ on. Like active labeling, the active voting is only possible if the system can bootstrap a classifier on preexisting labeled examples. In early stages, most matches are mostly ‘No’ with a small number of ‘Yes’.
• Prompt User - Jaxon will provide the user with a label and ask the user to create a completely novel example that matches that label to be added to the dataset.