Skip to content

Example: Generating a Report

Jump to bottom
Mads Hansen edited this page Jun 8, 2017 · 6 revisions

Throughout the remaining examples we will use the properties file approach to configuring CORB. It's a clean way to illustrate the difference between which properties you should use for different use cases. Executing CORB using a properties file is the same regardless of what properties it contains. For all remaining examples, CORB can be kicked off using the following command:

java –cp "marklogic-corb-2.3.2.jar: marklogic-xcc-9.0.1.jar" \
  –DOPTIONS-FILE=corb.properties \
  com.marklogic.developer.corb.Manager

Getting back to the task at hand, to generate a report that is written to the file system will require the use of more properties. But while we're at it, let's put some headers and footers in the report so in addition to the values given in the first example we’ll add:

  • Name of PROCESS MODULE: generate-report.xqy
  • Report file name: CORB_Report
  • Report file location: /data/MarkLogic/Uploads/
  • Report headers: ApplicationId, PolicyId, Cost
  • Report footers: ,,Total (only one footer in the third column)

Using a Selector Module

We will use the same selector module as the previous example to determine which documents we want to report on. Remember, all modules must be in the database at the URI specified which in this case is at the root (/). The corb.properties file must be updated as follows:

XCC-CONNECTION-URI= xcc://ffe-user:ffe-user$pwd@localhost:9500/FFE
URIS-MODULE= get-uris.xqy 
PROCESS-MODULE= generate-report.xqy 
THREAD-COUNT=8 
MODULES-DATABASE=FFE-Modules
PROCESS-TASK=com.marklogic.developer.corb.ExportBatchToFileTask
EXPORT-FILE-NAME=/data/MarkLogic/Uploads/CORB_Report.csv
PRE-BATCH-TASK=com.marklogic.developer.corb.PreBatchUpdateFileTask
EXPORT-FILE-TOP-CONTENT= ApplicationId, PolicyId, Cost
POST-BATCH-TASK=com.marklogic.developer.corb.PostBatchUpdateFileTask
EXPORT-FILE-BOTTOM-CONTENT=,,Total

Using a Data File as a Selector

This time, instead of using XQuery to select documents for reporting against we will use an input file which contains the URIs. This can be accomplished easily by making the two changes below to properties:

  • Remove the property, URIS-MODULE
  • Add the property, URIS-FILE=myLocalFileWithSelectorInformation.csv

Note: An input file can have one or more values per line. Each line will be read separately and sent to the Transform module. If multiple values are included per line, it’s up to the Transform module to parse the values accordingly.

Using ADHOC Queries, Passing External Variables to Modules and Zipping the File

Now we'll go back to using XQuery to select documents but we won't load it, or the transform module, into the database. Instead, we'll use the CORB 2.1.0 ADHOC feature to read them from the file system. We'll also pass in values for custom variables to the Transform module for specifying a date range. After the report is created, we'll use the POST-BATCH-TASK PostBatchUpdateFileTask to zip the file before saving it to the file system.

The corb.properties file must be updated as follows:

XCC-CONNECTION-URI=xcc://ffe-user:ffe-user$pwd@localhost:9500/FFE
URIS-MODULE=get-uris.xqy|ADHOC
PROCESS-MODULE=generate-report.xqy|ADHOC 
PROCESS-MODULE.startDate=20150301
PROCESS-MODULE.endDate=20150323
THREAD-COUNT=8 
MODULES-DATABASE=FFE-Modules
PROCESS-TASK=com.marklogic.developer.corb.ExportBatchToFileTask
EXPORT-FILE-NAME=/data/MarkLogic/Uploads/CORB_Report.csv
POST-BATCH-TASK=com.marklogic.developer.corb.PostBatchUpdateFileTask
EXPORT_FILE_AS_ZIP=true

Using Encryption

As of CORB 2.1.0, encryption is supported for the XCC-CONNECTION-URI, XCC-USERNAME, XCC-PASSWORD, XCC-HOSTNAME, XCC-PORT and XCC-DBNAME. The following examples assume that the encryption has already occurred as was specified in the section entitled, Encryption.

To Update A Node Using Encryption Supported by the CORB PrivateKeyDecrypter Class

In this example, it's assumed that the database connection string has been encrypted using either Java Crypt, RSA Keys or ssh-keygen. Additionally, encryption algorithm chosen was RSA which is the default.

To update a node, using XQuery for the selector and passing collection name to the transform query, the corb.properties file would look like:

# encrypted URI value
XCC-CONNECTION-URI=ljjlj3k4l234ll234lk23j4jl2jll33k44;l23k44j3l42j34
URIS-MODULE=get-uris.xqy 
PROCESS-MODULE=update-node.xqy 
THREAD-COUNT=8 
MODULES-DATABASE=FFE-Modules
DECRYPTER=com.marklogic.developer.corb.PrivateKeyDecrypter
PRIVATE-KEY-FILE=/path/to/private.key
PROCESS-MODULE.collectionName=updatedDocumentsCollectionName

To Generate a Report Using Encryption Supported by the CORB JasyptDecrypter Class

In this example, we'll use the JasyptDecrypter class to decrypt a password encrypted by Jasypt while running a CORB job to generate a report. It's assumed that the password will already be encrypted following the steps outlined in the section entitled Encryption. The URIs will be provided by an input file called uris.csv. The following properties will be used:

XCC-USERNAME=FFEUser
XCC-PASSWORD=sdflkjlj3l4j3j4l32jk4l32kj423432423423jlkjlk;jlkjj
XCC-HOSTNAME=localhost
XCC-PORT=8090
XCC-DBNAME=FFE
URIS-FILE=uris.csv
PROCESS-MODULE=generate-report.xqy
THREAD-COUNT=8
MODULES-DATABASE=FFE-Modules
PROCESS-TASK=com.marklogic.developer.corb.ExportBatchToFileTask
EXPORT-FILE-NAME=/data/MarkLogic/Uploads/CORB_Report.csv
DECRYPTER=com.marklogic.developer.corb.JasyptDecrypter
Clone this wiki locally