Processors Overview
Processors (also known as Job Tasks) are objects used to define a unit of work that will be performed against all documents in a job queue. The primary method of the processor is appropriately, process(). This is the method called by the migration manager to execute the work.
A processor will return the status of each document it processes. The creator of a processor can dictate how a document is treated when it leaves the processor. For example, if the processor deems the document failed, it can return a failed status. This means that the job will show a failure for that document due to the processor. If a document's status is set as failed or skipped in a processor, subsequent processors in the job's processor list will not be run.
Simflofy comes packaged with several processors and post-processors out-of-the-box. The work a processor performs varies and includes things like:
- cleaning up file names
- removing duplicates
- pulling supplemental data from external systems
- content transformations
Complete List Of Simflofy Processors
Building a Processor
We designed processors to be as simple as possible for extension. They are a powerful tool that can significantly improve the quality of the data you are migrating, indexing, or analyzing.
Developing processors requires a working knowledge of Java and Spring. If you do not wish to delve that deep, try putting your logic in a JavaScript Processor Task.
When developing Processors for Simflofy an important thing to keep in mind is the impact your work will have on the overall performance of the migration job. Each document you process will be impacted by your processor.
Mitigate performance issues by:
- Leveraging Init and Teardown methods properly
- Caching where applicable
- Understanding JVM and memory management
- Carefully plan calls to external systems to limit processing time
Sample Processor Code
Code for this sample processor can be found on our sample-code project on GitHub.
If you have problems with the code, please open an issue.
Contact Simflofy for support