Skip to content

ml gradle 1 tasks

Jump to bottom Edit New page
rjrudin edited this page May 9, 2017 · 11 revisions

The tasks below only appear in ml-gradle 1.* releases. That version of ml-gradle uses the Packaging REST API, and thus most of these tasks involve packages. ml-gradle 2.* and beyond use the Management REST API and thus do not have these tasks.

Deploy an application

The mlDeploy task combines several tasks together to perform a typical deployment process. Those tasks are:

  • mlClearModules - if the application exists, clear its modules database; otherwise do nothing
  • mlInstallPackages - installs the application packages
  • mlPostInstallPackages - an empty "hook" task for adding tasks via its dependsOn attribute
  • mlLoadModules - loads all of the modules
  • mlPostDeploy - an empty "hook" task for adding tasks via its dependsOn attribute

You don't need to use mlDeploy - you're free to write your own Gradle task to orchestrate your deployment process. mlDeploy exists as a useful starting point that you can extend via the mlPost* tasks that it calls.

Install application packages

gradle mlInstallPackages

This task corresponds to the InstallPackagesTask.groovy class, and it is installed by MarkLogicPlugin. It provides a basic recipe for installing a typical set of MarkLogic resources:

  1. A database and optional test database is installed (with the forests being automatically installed). These databases can optionally be defined via a database package file, which is a property of the ManageConfig object that is inserted under "mlManageConfig".
  2. A REST server and optional test REST server is installed.
  3. An XDBC server and optional test XDBC server are installed. These are mostly intended to support operations involving ingesting data (e.g. via MLCP) and fiddling with the database in a JUnit test.

You can run this task repeatedly without breaking anything.

You typically won't call this directly, but rather you'll call "gradle mlDeploy" which will call this task.

Reasons why this task would fail:

  1. ML will reject invalid package files. If you try to install a bad one, the ML error log should have a helpful error message for you.
  2. If your package has a server package file in it that tries to use a port already in use, the install will fail.
  3. "mlUninstallApp" has a bug in it where the forest backing the content database isn't always deleted. If you run "mlUninstallApp", and the forest isn't deleted, and then you run "mlInstallPackages", you'll get another helpful error message. In this situation, you'll need to manually delete the forest via the admin UI.

Managing security resources

ml-gradle provides several tasks for managing security resources - users, roles, and amps. No support exists yet for creating many of these via, for example, a single XML file; that support is planned for the future. Instead, these tasks are used by declaring new tasks in your build file that have a type of one of these tasks.

For example, you can create a user:

task createUser(type: com.marklogic.gradle.task.security.CreateUserTask) {
  username = "sample-user"
  userDescription = "This is a sample user"
  password = "password"
  roleNames = ["rest-admin", "rest-writer"]
}

And you can create a role:

task createRole(type: com.marklogic.gradle.task.security.CreateRoleTask) {
  roleName = "sample-role"
  roleDescription = "This is a sample role that extends a couple pre-defined roles"
  roleNames = ["rest-writer", "rest-reader"]
}

Note that both of these tasks must prefix their "description" property with "user" and "role" so as to avoid conflicting with the built-in "description" property on a Gradle task.

Each of these tasks supports additional task properties for customizing users and roles - see the source code for more information.

Merging database packages

gradle mlMergeDatabasePackages

The purpose of this task is to merge two or more database package files together. This provides an easy way of sharing package files and avoiding duplication between them.

This task requires a list of file paths to be provided. It then uses "default-content-database.xml" as a starting point, and then uses "content-database-transform.xsl" to merge in each of the packages at the given file paths (both of these files live in the ml-app-deployer project). Since this task is registered by MarkLogicPlugin, you can configure it in the following manner:

ext {
  mlAppConfig {
    databasePackageFilePaths = ["package1.xml", "package2.xml"]
  }
}

You typically won't invoke this task directly - that's because all it does is produce a merged file. Instead, you'll want other tasks to depend on this task - e.g. any tasks that will install the database package. By default, MarkLogicPlugin makes mlUpdateContentDatabase and mlInstallApp depend on this task.

Merging HTTP server packages

See the writeup on merging database packages above - this is nearly identical, except for the following:

  1. This is implemented by MergeHttpServerPackagesTask
  2. The default HTTP package file is "default-http-server.xml"
  3. The XSL is "http-server-transform.xml"

You can configure the server package files to be used like this:

ext {
  mlAppConfig {
    httpServerPackageFilePaths = ["package1.xml", "package2.xml"]
  }
}

One other note - if you intend to customize your HTTP server, you'll almost always needs to ensure that "mlUpdateHttpServers" is invoked as part of your deployment process. The REST API for creating REST API servers doesn't allow for a package file to be specified. Thus, you'll normally have the following sequence:

  1. Create a REST API server
  2. Create a merged HTTP server package file (via this task)
  3. Upload the merged package file and re-install your MarkLogic package (via mlUpdateHttpServers)

Adding mlUpdateHttpServers to mlDeploy can be done via the mlPostDeploy hook:

mlPostDeploy.dependsOn mlUpdateHttpServers

Uninstalling an application

gradle mlUninstallApp

This is implemented by UninstallAppTask.groovy, and it's registered by MarkLogicPlugin.

This task uses the XDBC server installed by InstallPackagesTask to execute an XQuery statement that removes all of the app servers, databases, and forests installed by InstallPackagesTask. To make this all work, this task first assigns all of the app servers to the Documents database, thus allowing the underlying databases and forests to be deleted. App servers are also deleted last so that the server restart doesn't occur until the very end.

Updating HTTP servers

gradle mlUpdateHttpServers

This task is implemented by UpdateHttpServerTask.groovy, and it is registered by MarkLogicPlugin.

This task is useful for when you want to modify the HTTP server that's created when mlInstallApp creates a new REST API server. Some common reasons for modifying it:

  1. You want to specify a different rewriter or error handler
  2. You want to specify a default user
  3. You want to change the authentication scheme

Unlike mlUpdateContentDatabases, which you may run somewhat frequently as you're making changes to your database configuration, you probably will not run this task directly very often. Instead, it's more likely that you'll include this in the "dependsOn" attribute of "mlPostDeploy". That's because you can't control the HTTP server configuration when mlInstallApp is creating a REST API server, and thus, you'll need to modify it after you've installed.

Updating the content databases

gradle mlUpdateContentDatabase

This task is implemented by UpdateDatabaseTask.groovy, and it is registered by MarkLogicPlugin. It is intended to be executed after you've made a change to the package configuration for your content database. It simply adds the database package to the application package and then re-installs the package.

By default, as specified in ManageConfig, this task expects to find a database package at src/main/xqy/packages/content-database.xml. This is often overridden though to take advantage of merging database config files together.

Managing CPF resources

ml-gradle provides several tasks for managing CPF resources. These tasks allow you to perform the steps necessary for utilizing CPF:

  1. Creating a domain
  2. Creating a domain configuration
  3. Inserting a pipeline

Let's first look at setting up CPF to support MarkLogic Alerting. We first need to create a domain for documents that may generate alerts. Let's do that for the "sample" collection:

task createRevisionDomain(type: com.marklogic.gradle.task.cpf.CreateDomainTask) {
  domainName = "sample"
  domainDescription = "The sample collection"
  scope = "collection"
  scopeUri = "revision"
}

Next, we need to create a domain configuration. We'll use our sample domain as the default domain:

task createRevisionDomainConfiguration(type: com.marklogic.gradle.task.cpf.CreateDomainConfigurationTask) {
  restartUser = mlAppConfig.username
  defaultDomainName = "sample"
}

Note that in this case, we reuse the username defined in mlAppConfig. You can of course customize this to be any MarkLogic username. Also, this task sets a couple default properties. It assumes that the evaluation context is a modules database named after the mlAppConfig.name property - e.g. if the name of your application is "myapp", then it assumes the modules database is "myapp-modules". It also sets default permissions of app-user:read and app-user:execute.

MarkLogic Alerting requires the Status Change Handling and Alerting pipelines, so we'll handle those next. ml-gradle has a special task - InsertSystemPipelineTask - for inserting system pipelines - i.e. pipelines included in MarkLogic. And because it's common to need to insert these two pipelines, ml-gradle has a task for each, and each is registered by ml-gradle under the names "mlInsertSchPipeline" and "mlInsertAlertingPipeline". So in this case, we don't need to create any new tasks. We can just call these tasks via Gradle or, more likely, attach them to another task via the Gradle dependsOn parameter.

Let's say we instead want to insert our own pipeline. To do, we'll use InsertPipelinesTask, which expects to read pipeline XML files from the file system. Typically, you'll put a pipeline in your project directory. There's no required location, but let's say we have a "my-pipeline.xml" file under "src/main/xqy/cpf". Here's how we'd insert this pipeline:

task insertMyPipeline(type: com.marklogic.gradle.task.cpf.InsertPipelinesTask) {
  filePaths = ["src/main/xqy/cpf/my-pipeline.xml"]
  domainNames = ["sample"]
}

Note that we can specify multiple file paths and multiple domain names. Each pipeline will be attached to each domain. If you do not want this behavior, just create multiple instances of this task class.

To clean up CPF resources, ml-gradle currently just supports removing domains via RemoveDomainsTask:

task removeDomain(type: com.marklogic.gradle.task.cpf.RemoveDomainsTask) {
  domainNames = ["sample"]
}

ml-gradle will soon support removing all CPF resources - domains, domain configurations, and pipelines.

PDF Conversion Example

//----------------------------------------
// CPF Setup

task mlInsertSystemPipelines(dependsOn: ["mlInsertSchPipeline"], group: "CPF")

task createPdfDomain(type: com.marklogic.gradle.task.cpf.CreateDomainTask, group: "CPF") {
  domainName = "pdf-domain"
  domainDescription = "PDF processing"
  scope = "collection"
  scopeUri = "pdf"
  pipelineNames = ["Status Change Handling", "PDF Conversion (Paged Text, No Rendering)"]
}

task createPdfDomainConfiguration(type: com.marklogic.gradle.task.cpf.CreateDomainConfigurationTask) {
	restartUser = mlAppConfig.username
	defaultDomainName = "pdf-domain"
}

task insertPdfPipeline(type: com.marklogic.gradle.task.cpf.InsertSystemPipelineTask) {
	filename = "Installer/conversion/alternatives/paged-text-pdf-pipeline.xml"
}

task removeDomain(type: com.marklogic.gradle.task.cpf.RemoveDomainsTask) {
	domainNames = ["pdf-domain"]
}

task installCpfResources(dependsOn: ["mlInsertSystemPipelines", "createPdfDomain", "createPdfDomainConfiguration", "insertPdfPipeline"], group: "CPF")
createPdfDomainConfiguration.mustRunAfter createPdfDomain
createPdfDomain.mustRunAfter mlInsertSystemPipelines
createPdfDomain.mustRunAfter insertPdfPipeline
Add a custom footer
Add a custom sidebar
Clone this wiki locally