In every area of IT systems and devices generate plethora of data. Turning this data in to informative and actionable is the key. Data is heterogeneous, complex and contain wealth valuable information. Message generated by various application/devices has varying degree of message signature. Message may be well formatted or verbose plain English sentence. Knowledge Object will perform extraction of information in log search which and is known as” Interesting Tokens”.
Knowledge Object is collection of ‘Event Rules’ and ‘Expressions’ under these rules. Important key definitions are as below
Event Rule:
‘Event Rule’ can be defined by assigning at least one Standard property [Log Type, Event Source, Event ID and Event Type] and also specifying expression in description [both Inclusion and Exclusion]. Description can be specified either by Regular Expression or text. Following are some constraints while creating rule:
· Specify at least one of the Properties and Description pattern to recognize uniquely.
· May contain more than one ‘Event Rule’ per ‘Knowledge Object’.
· There should not be any ambiguity in constraint which could match to more than one ‘Knowledge Object’
Message Signature:
Message signature is part of ‘Event Rule’ and is a distinct pattern to identify message. In EventTracker logs are stored in database with certain degree of normalization and schema. A log can be divided in to two sections i.e. ‘Properties’ and ‘Description’. Together these can be part of the signature. Message signature can contain matching or exception message pattern and may also contain any of the event properties.
Event rules and Ambiguity:
Each Knowledge Object constitutes of ‘Event Rule’ and ‘Expressions’ for extracting valuable information. One of the most important factors about ‘Event Rule’ definition is that there should not be any ambiguity in identifying ‘Knowledge Object’. To achieve this, every ‘Event Rule’ must be unique and should not resolve to more than one ‘Knowledge Object’. In case a log matches more than one Knowledge Object, weightage based algorithm will decide the ‘Knowledge Object’ that has to be applied to the target log and will extract interesting tokens accordingly.
Expressions:
Expressions are defined under each rule. ‘Expression’ is used to extract atomic information from the message. One rule can have zero or more expressions. If no expressions are specified, search engine will try to match with well-known signature and extract information.
Message signature recognition can be helpful in extracting ‘Key’ and ‘Value’ pairs from the message if any. Some signature will have definitive Key-Value Pair (KVP) pattern where it is sufficient just to provide Key and Value delimiter for e.g. in windows messages KVPs are recognized by ‘:’ (colon) and ‘\r\n’ (new line). In many cases this KVP may be found with various Key and Value delimiter. But sometimes it is not possible to define any KVP because the message is in plain verbose English. In these cases there will not be any key to specify value. Generally messages can be identified having:
· Well defined KVP with single Key and Value delimiter
· Well defined KVP with multiple Key and Value delimiter.
· Messages with well-known information pattern (IP, URL, filenames, dates etc.) but missing KVP
· Verbose plain English sentence where neither KVP nor well-known information pattern exists.
Based on above observations ‘Expression’ can be of following type
· Key Value delimiter
· Regular expression (Requires virtual column name depending on Regular Expression)
· Column delimiter (Requires virtual column name)
· Between 2 constant string (Requires virtual column name)
Knowledge Objects consists of 2 panes namely action pane and editor pane.
Action Pane: This pane will have list of all available Source types and can be viewed and edited by clicking one of the Source types
Editor pane: This is the pane where are CRUD operations, Validations and Verification of constraint and Expressions can be carried out.
User can now categorize the knowledge objects by creating customized groups and adding Knowledge Objects to the group, as per requirement.
1. Login to EventTracker Enterprise.
2. Click the Admin menu, and then click Knowledge Objects.
Knowledge Objects page displays.
The Knowledge Objects will be listed.
3. To add a group, click 
icon in the GROUP pane.
Add Group window displays.
4. Enter the Group name: and Description.
NOTE:
Ø The Group name is unique.
To activate the group and adding Knowledge Objects to it, make the required changes using Edit feature and it is explained below.
a. Click 
icon in the GROUPS pane for editing the group.
Edit group window displays.
1. To add an object to a group, click 
icon in the right hand corner.
Add object window displays.
2. Enter the Object name:, Applies to:, Object version:, Description.
NOTE:
The Object name is unique.
3. Select the group from the dropdown box, where you wish to add the object.
4. Click Enabled option.
5. Click the Save button.
NOTE:
The object added to the group can be edited and can also be enabled or disabled.
To activate this data in the Knowledge Object, make the required changes using Edit feature and it is explained below.
b. Click 
icon for editing the Object.
Edit object window displays.
c. Select Enabled option, and if required make the necessary changes.
d. Click the Save button.
To add Default Knowledge Objects to the Custom Group,
1. Expand the Group.
2. Select the Knowledge Object and click the edit 
icon.
For Example: Cisco Meraki AccessPoint Association
Edit Object window displays.
3. Select the custom group.
Here we selected “Test Group”.
4. Click the Save button.
The Object gets added to the group.
The user will now be able to search a Knowledge Object by typing the name of the object in the search box and clicking the search icon
.
Objects are available to import or export in a JSON format, file with extension .etko.
For Exporting Objects,
· Click on the Admin> Knowledge Objects.
· Expand the respective group and select the Knowledge object you want to export and click on the export 
icon.
· EventTracker:: Knowledge Object Import/Export page gets displayed.
· Select the object by clicking on the check box and click Export.
1. To import objects, click Import 
icon.
Import window displays.
2. Click the Browse.. button, and then select the location of the file.
NOTE:
· The file to be uploaded should be in ‘.etko’ format only otherwise an error message is displayed as shown below.
3. Click the Upload button.
4. Select the Object name option.
NOTE: If the Object name is not selected an error message will be displayed. Click OK.
5. Click the Merge/Overwrite button.
This option allows to activate/deactivate the default Knowledge Objects that have been provided. It is not possible to edit other details of default Knowledge Object. The user defined/created Knowledge Objects can be edited/updated.
1. To edit the Knowledge Object, click Edit icon.
Edit object window displays.
2. To disable the Knowledge Object, select the Enabled option and then click the Save button.
|
2 NOTE |
|
However, you don’t have an option to edit and remove the Pre-Defined Knowledge Objects |
1. Expand an existing Knowledge Object group and select the object.
Ex: F5 BIG IP LTM Configuration
2. Click Add Rule 
icon.
EventTracker Add Rules page displays.
3. Enter Title, Log type, Event type, Event source, Event id, Message signature, Message exception, Sample message.
4. To verify message signature and exception, click Verify signature & exception 
icon.
5. To perform a log search, click Log search 
icon.
6. To add a regular expression, click Add new expression icon.
NOTE: Without Sample message, you cannot add expression type.
|
Expression type |
Description |
|
Key Value Delimiter |
Enter Key delimiter and Value delimiter. Click the Add button. To test the key value delimiter, click the Test button. Examples of Key Value delimiter is colon (:), hyphen (-), space (/s), newline (\n) etc.
|
|
Regular Expression |
Enter a Regular expression, select Expression level and then click Add.
If ‘Expression level’ is selected as ‘Root’, then entire regular expression is considered to extract the values. If Expression level is selected as Group, then the regular expression is split into smaller groups for extraction. If Format string has not been provided. To test the Regular Expression, click the Test button.
|
|
Between Two Strings |
Enter Format String, Left String, Right String. If required to test the expression, click the Test button. Click the Add button. For example:
|
|
Column Delimiter |
Enter column Delimiter and then click the Add button. To test the column delimiter, click the Test button. For example:
|
The predefined Knowledge Objects cannot be deleted. The Knowledge Object created by the user can be deleted by him/her only.
1. To delete a knowledge object, click the 
icon.
Message from webpage displays:
2. Select the OK button.
Let us consider an example to add IP Address as a Knowledge Object.
1. Click Add Object icon.
2. Enter the relevant data; click Enabled option to activate the Knowledge Object.
3. Click the Save button.
4. Click Add Rule 
icon.
5. Enter Title, Event source, Message signature, and Sample Message.
If Log type, Event type, Event id, Message exception is known, then enter the information.
6. Click Add new expression icon.
Add/Edit expression window displays.
7. Select Expression type and Expression level as required.
In our example, we are selecting Regular Expression as Remote Address:\s[\d.]+. Enter the Regular Expression, and then click Test.
Test results display.
8. Enter a name for Test Results box as it is the Format String.
9. Click the Add button.
10. To edit the configured expression, click the Edit 
icon.
11. To delete the configured expression, click the Remove 
icon.
12. Click the Save button.
13. In Rules pane, click Verify Signature 
icon.
14. In Expressions pane, click Verify Signature icon.
Sample message verification results display.
EventTracker v9.2 introduces optimized Knowledge Objects search and improved mapping capabilities to speed search performance.
Conditional tagging helps to users to perform efficient log search across all log sources.
With EventTracker v9.2, the user can now use Configured Conditional tags to identify the values which they are looking in log search result from across all the log sources (Knowledge Objects -KOs).
The below Knowledge Objects have the updated conditional tag configuration.
Accessing the Conditional Tag Configuration
1. Log into EventTracker, go to Admin and select knowledge objects.
Knowledge objects page opens.
2. Click on Groups to expand, here Checkpoint Firewall is taken as an example, and then click on the object CheckPoint Firewall System.
CheckPoint Firewall System pane is displaced on the right side.
3. Click on the CIM Mapping icon , located at the right top corner of the right pane.
CIM Mapping page opens.
4. Click on the Configure Conditional Tag(s) icon.
5. Conditional Tag Configuration page opens.
You can search by CIM Field and value criteria. Accordingly, the results are displayed.
CIM Fields available in knowledge objects are shown.
1. In the Search by field, let us choose CIM Field and choose the log_type and then click search.
2. All the CIM Fields related to log_type results are displayed.
3. Choose Value Criteria in the Search by and type in the value criteria clish*.
4. The results with CIM Field log_type and value criteria clish* is displayed.
