All Classes and Interfaces
Class
Description
This class is the entry point for processing operations.
This method is equivalent of TransitionEntity in the library module.
Mock durations for different Actions.
Performs the Action and maintenance around the Action such as
- Setting the start time and saving the operation's duration
- Executing pauses before and after the operation
- Determining the success of the operation
- Saving the MatchSnapshots
Any Action that is run independently of this class should
have code to perform maintenance around the Action.
Actions that are run from the Action class need to follow this interface.
Interface for creating different types of logs.
ActionOptions provides options for configuring an action.
Utilities for serializing ActionOptions to and from JSON.
Custom serializer for ActionOptions to handle special cases
An ActionResultsCombos has both an action and an expected result.
Returns the corresponding Action class as specified in ActionOptions.
Determines the text description of an action given the action itself as well as the active states before
and after action execution.
An ActionVector is the translation of the ActionOptions variable into a vector of numbers,
to be used for machine learning.
This translation method uses one hot encoding to encode the type of action performed.
There can be different ways to translate an ActionOption variable to a vector.
Non-image objects are adjusted and added directly to the Matches variable.
Adjust Matches according to the options in ActionOptions.
After a successful click,
Sets the probability that the object remains visible or disappears.
Manages the State repository.
This class is a central location for pixel analysis, for every color-based match strategy.
Anchors define
1) A Position in the Image Match or Region
2) A Position in the Region to define
A point defined with Position #1 is used to define part of a Region.
Gets the window of the App in focus.
Validates automation DSL JSON against the automation DSL schema.
Represents the automation configuration for UI buttons in the Brobot Runner.
BasicActions, which run for 1 iteration, require 1 or no Find operations.
Manages Brobot's runtime environment configuration.
Builder for creating custom BrobotEnvironment configurations.
Mixin for Brobot Image class to prevent serializing BufferedImage.
Global settings
Some Brobot settings override Sikuli settings.
This class implements the SmartLifecycle interface to perform actions after
the Spring context is initialized.
Mixin for BufferedImage to prevent serializing internal classes.
Main entry point for business rule validation.
Represents a UI button configuration for the desktop runner that calls automation functions.
Represents position information for a button in the UI.
Represents styling information for a button.
Captures the screenshot from the screen where the mouse is located and saves it to the given directory.
Saves a series of screenshots.
DynamicImages classify as probabilities, standard Images have 100% pixel probability when found.
Clicks on an Image Match, Region, or Location.
Wrapper class for Click, handles real and mock clicks.
Converts ClickType to a Sikuli Button.
ClickUntil clicks Matches until a condition occurs or the operation times out.
When working with color profiles, images should be created that represent the colors
to be found.
ColorInfo has the min, max, mean, and stddev for a single ColorValue, such as Hue, Saturation, or Value.
Mixin for ColorModel to prevent serializing internal classes.
A ColorSchema is a representation of a color cluster.
ColorStatProfile is a color in a specific color space (HSV, BGR, etc.).
It is useful to create one or more CommonActions classes for your applications
in order to reduce code redundancy and make your code more readable.
Currently ActionResultCombos are set up for click operations as the first action.
This class defines some typical configurations for Drag and
allows for multiple drag operations.
This is an example of a more complex custom action.
This class should make it simpler to create common StateTransitions.
CompositeActions are built from BasicActions.
Exception thrown when there is an error in processing Brobot Runner configurations.
Exception thrown when configuration validation fails.
Main validator class for Brobot configurations.
Performs nested Find actions, in the order they appear in the FindActions list in
ActionOptions.
This class runs an automation script continuously by
- searching for active states at a fixed interval
- running transitions for any active states found
When using this script, the model could contain some states without transitions and some states with transitions.
This class creates a deep copy of most of the fields in ActionOptions.
Mixin for DataBuffer to prevent serializing internal classes.
Each region has 4 borders that can be defined: top, bottom, left, right.
Helper functions for various Define classes
Defines a Region as the largest rectangle including all Matches and Locations.
Defines a Region as the smallest rectangle produced by Matches and Locations.
Defines a Region as the largest rectangle produced by Matches and Locations.
Defines a Region with the Define method specified in the ActionOptions.
Defines a Region around a Match.
Defines a Region using the active Window as a reference.
Completes a whole Transition (FromTransition and ToTransition),
including all States to activate and exit.
The DoUntilActionObject contains both ActionOptions and ObjectCollections for both
'do' and 'until' Actions.
Drags from an Image Match, Region, or Location to an Image Match, Region, or Location
Drag is a Composite Action composed of a Basic Action (Find) and a Sikuli Wrapper (Drag Location)
Wrapper class for Drag, handles real and mock drags.
Last used in version 1.0.6.
IllustrationScenes store their filenames before they are written to file.
Brobot StateImage objects can contain multiple patterns.
Finds all Matches for all Patterns in the Brobot Image.
The steps for color classification are:
1.
Retrieve a Find function with its Find type.
Searches a scene for pattern and word matches.
TimeBetweenIndividualActions is the option in ActionOptions that gives the time (in seconds)
between the first, second, and third screenshots.
Sends the Image object to either FindImage or FindRIP depending on whether the Image location can vary.
Finds the regions with dynamic pixels.
The image to compare is in ObjectCollections #1 (0), in the list of StateImage objects.
Validates references related to automation functions, including function calls,
parameters, and references to functions from other components.
Validates business rules related to automation functions.
Returns the 'from' and 'to' Locations to be used in Drag.
The same methods as in GetImageOpenCV, but for JavaCV.
This class is primarily for color analysis.
Scenes are used for
- finding color
- image segmentation (classification)
- action illustration
In Brobot version 1.0.7, scenes are used for all Find operations.
Wrapper class for GetText, handles real and mock text queries.
Grid is either defined by columns and row, or by cell width and cell height.
Deprecated.
Highlight all Matches at once, or one at a time.
Wrapper class for Highlight, performs real or mock highlights.
Has the mask for one part of each image, i.e.
Divides a region into five subregions to facilitate histogram and color Find operations.
The physical representation of an image, stored as a BufferedImage, sent to the database as a byte array, and
retrievable in these forms as well as a JavaCV Mat of BGR or HSV.
Custom deserializer for Image class that creates an Image with an empty BufferedImage.
Validates that image resources referenced in the project configuration actually exist
and are valid image files.
Custom serializer for Image class that omits the BufferedImage.
Given sets of possible active States, this class searches for these States
to set active States.
After a successful transition (both 'from' and 'to' Transitions):
'activate' holds all States to make active
'exit' holds all States to deactivate
Configuration for JSON processing in the Brobot application.
JsonParser provides methods for working with JSON in a robust way.
Utility class for working with JSON paths and extracting data from complex JSON structures.
Utility class that provides additional JSON functionality focused on
handling problematic Java classes (circular references, module system issues).
Uses only ObjectCollection #1
An ObjectCollection can have multiple keys
ActionObjects hold special keys such as CTRL that are pressed first
KeyDown in SikuliX (Java Robot) does not actually hold the key down.
Uses only ObjectCollection #1
An ObjectCollection can have multiple keys
ActionObjects hold special keys such as CTRL that are released last
Wrapper class for KeyUp, works for real or mocked operations.
KmeansCluster represents a cluster of points in a k-means analysis corresponding to one of the centers.
KmeansProfile represents a k-means analysis for one number of means and one color schema (BGR, HSV, etc.).
Creates a series of kmeans profiles for commonly used number of centers (2-10)
Calculating this up-front will save time and CPU/GPU.
Location is calculated
- when the region is defined, first with the position in the region, then with the offset
- when the region is not defined, with the x,y values, then with the offset
An interface for components that wish to persist log data produced by the library.
This builder puts a bunch of Mats together.
The SikuliX Match object has many protected fields, making it difficult to work with as a field.
The results object for all actions.
Manages the Matches throughout the lifecycle of the action.
Utilities for serializing Matches to and from JSON.
Custom serializer for Matches to handle special cases
Match objects do not retain data on Pattern, StateObject, etc.
Used for mock runs when not empty; otherwise, the State's probability variable is used.
Creates Match objects when probability is being used instead of Snapshots.
Mixin for Match class to prevent serializing fields that could cause null reference issues.
MatchSnapshots record a match (or failed match) and the search options at a single point in time.
3d versions of common OpenCV operations that only accept one channel.
Custom serializer for Mat objects
Mainly for testing, this class provides code to easily visualize Mats.
Returns snapshots with matching actions and states.
Takes care of setting probabilities for States and State objects
Keeps track of the number of mocks performed.
Mock text for GetText Actions using Snapshots.
Keeps track of the mocked value of 'now' and performs mocked pauses.
Presses and holds a mouse button.
Wrapper class for MouseDown, works for real or mock actions.
Releases a mouse button.
Wrapper class for MouseUp, works for real or mocked actions.
Wrapper class for MouseWheel, works for real or mocked actions.
Moves the mouse to one or more locations.
Wrapper class for MoveMouse, works for real or mocked actions.
Helper function for Composite Actions
NOT WORKING! Sikuli appears not to allow mouse movement after a mouse down operation.
This is an example of how to create an Action using the MultipleActions class.
Performs nested Find actions, in the order they appear in the FindActions list in
ActionOptions.
A class representing no match.
A default, do-nothing implementation of LogSink.
NullState is used as a generic State for passing simple objects
(non State objects) to Actions.
This class holds all the objects that can be passed to an Action, which include:
- StateLocation
- StateImage
- StateRegion
- StateString
- Matches
- Image: these are scenes, or screenshots on file, that are used for the operation instead of the active environment
(what is currently on the monitor).
Utilities for serializing ObjectCollection to and from JSON.
Custom serializer for ObjectCollection to handle special cases
ParameterCollections help Brobot calibrate specific parameters with respect
to success conditions.
A single path from a start State to a target State.
Finds all paths from a set of start States to a target State.
If a Path is unsuccessfully traversed, we find ourselves somewhere in the middle of
the Path.
A list of Path objects comprising the possible ways to go from
start States to the target State.
Pattern and StateImage_ (to replace StateImage) are for a code restructuring that gets rid of RegionImagePairs.
Holds the results of all per-pixel color analysis for a {scene, ColorProfile} pair.
Contains a series of PixelAnalysis objects that comprise all analysis of a {scene, StateImage} pair.
Position is defined by (%w, %h) and can be used to get a location in a Region or Match.
Represents the project configuration settings and references to automation functions.
Validates project configuration JSON against the project schema.
It divides a region into
grid cells, some overlapping, and counts the points that are within the cells.
This class is meant mostly for testing state structures and state management,
although it can also be used for bootstrapping simple projects.
Mixin for Raster to prevent serializing internal classes.
Mixin for java.awt.geom.Rectangle2D to prevent possible circular references.
Custom serializer for JavaCV Rect objects to prevent circular references.
Main facade for cross-reference validation between configuration entities.
Region uses methods from the SikuliX Region and adds new initializers and analysis tools.
Mixin for Region class to prevent circular references.
Prints to the console if the output meets the required reporting level.
Returns a List with the first element the Action Matches and the second element the Results Matches
ActionResultsCombos are typically:
- an Action such as CLICK
- a Result from an operation such as FIND
The parameters and results only get recorded if the Action is successful.
A collection of PixelAnalysisCollection objects, each of which contains all analysis of a {scene, StateImage} pair.
Records all scenes associated with an image, as found in SceneCombinations.
Manages JSON schemas for the Brobot Runner configuration system.
Entry point for schema validation operations.
Scrolls the mouse wheel up or down.
SearchRegions allows for multiple Regions to be associated with an Image.
Mixin for SearchRegions to prevent circular references.
Utility methods for working with SearchRegions objects.
Select is a custom Action built using the custom SelectActionObject.
Set the hidden States after a successful State transition.
Mats used to illustrate the image and its color profiles can be stored together with
the image and referenced for illustration when needed.
Mixin for SikuliLocation class to prevent circular references.
Mixin for SikuliScreen class to prevent circular references.
States give a Brobot application structure.
This class finds the active States after Brobot is lost.
States can be illustrated in various ways.
A collection of Pattern objects.
Represents a log entry for a state image.
A StateLocation belongs to a State and usually has a Location that
has a special meaning for its owner State.
StateMemory keeps track of which States are currently active.
The MatchHistory keep a record of the StateObject's Snapshots.
Provides the essential data fields to store a StateObject in other variables, like Match.
Validates references related to states, including state elements, transitions,
and references to states from other components.
A StateRegion belongs to a State and usually has a Region that
has a special meaning for its owner State.
A State String belongs to a State and usually has a String that
has a special meaning for the owner State.
StateText is text that always appears in its owner State.
StateTransitions hold all the Transitions for a State.
Finds the correct Transition from one State to another,
taking into account the hidden States.
Holds all static incoming and outgoing Transitions,
plus the variable transitions to hidden States.
We want to go to a specific State, the target State.
Manages the StateTransitions repository and retrieves
StateTransitions given a state name.
Calculates the similarity of 2 Strings.
Defines the success condition for individual Actions.
The TestManager is in charge of the progress of an automation application.
Tests can be used for mocks or for live execution.
Brobot's AUT testing strategy is to go through all states looking for missing
images or faulty transitions.
This interface manages the logging session.
Text read from the screen is a stochastic variable.
The TextSelector returns text from the stochastic Text variable (Strings are stored in a List),
which it believes has the highest probability of being the actual text on the screen.
This class returns an object containing the StateTransitions objects and StateTransition objects that
we are interested in for a specific transition from one State to another.
Validates business rules related to state transitions.
Moves through the Paths to reach the target State.
Aims to visualize the State hierarchy.
Types text to the window in focus.
Wrapper class for TypeText, works with real or mocked actions.
The Unknown State is usually the start point for any application,
and can also be found when Brobot has become lost.
Use the defined regions (if they exist) of objects as MatchObjects
Represents an error found during configuration validation.
Contains the results of a validation operation.
Represents the severity of a validation error.
Returns a successful Matches object if at some point no objects are found.
Mixin for WritableRaster to prevent serializing internal classes.
BrobotEnvironmentinstead.