-
Notifications
You must be signed in to change notification settings - Fork 77
Debugging guide
When running an ml-gradle task such as "mlDeploy", a number of things can go wrong - bad connection information, invalid username/password, bad resource files, bad REST modules, etc. This page intends to guide you through how to debug any errors that you run into before you have to ask for help.
Before going any further, run the task that just failed with Gradle's info-level logging enabled - let's assume it was mlDeploy that failed:
gradle -i mlDeploy
With info-level logging enabled, ml-gradle will log a number of useful things:
- Any overridden properties will be logged (except for passwords).
- Every command executed by mlDeploy will be logged, which includes the directory and files processed by the command.
- And most importantly, every call to MarkLogic - including the URL and the HTTP request body - will be logged.
When an error occurs, you can typically scroll up in the logging to see the last URL that was invoked by ml-gradle, along with the data it sent to MarkLogic. The error message from MarkLogic will also be logged. When that error message is cryptic, it may be worth reproducing the same request to the logged URL via an HTTP client like curl so that you can verify you get the same error when making a call directly to MarkLogic.
Whenever Gradle throws an error, it prints the following helpful information too:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
If the error message from info-level logging doesn't tell you exactly what the problem is, then follow Gradle's advice and include the "--stacktrace" option as well:
gradle -i --stacktrace mlDeploy
You can include "--debug" or ("-d") too, but it's not common that this will expose an ml-gradle problem that info-level logging and the stacktrace don't already show. And debug logging is very verbose, because you get all Gradle debug logging, not just ml-gradle's debug logging.
If you are connecting to MarkLogic with SSL/TLS, it's possible to run into a variety of errors due to a misconfiguration. Loading modules via SSL and SSL-with-Manage-and-Admin-servers describe how to configure ml-gradle based on your SSL requirements.
In the event of a bad resource file (resource = database, or user, or role, etc) - e.g. the file has malformed JSON or XML, or the contents of the resource file don't conform to the Manage API schema for that resource - the info-level logging above should identify the file that ml-gradle can't load.
In the case of malformed JSON/XML, ml-gradle should log an error that explains the issue if you include the "--stacktrace" option mentioned above. So if you have e.g. a malformed XML file for a role, you'll see logging like this:
gradle -i --stacktrace mlDeployRoles
... omitting some gradle logging ...
Execution failed for task ':mlDeployRoles'.
> Unable to read resource payload: null
... omitting more logging and a lot of the stack trace ...
Caused by: org.xml.sax.SAXParseException; lineNumber: 2; columnNumber: 1; XML document structures must start and end within the same entity.
In the case of a resource file that the Manage API can't process, the error thrown by the Manage API generally should explain the issue, and you'll get that issue without the "--stacktrace" flag too:
gradle -i mlDeployRoles
... omitting some gradle logging ...
Execution failed for task ':mlDeployRoles'.
> Unable to read resource payload: unexpected element (uri:"", local:"bad-role"). Expected elements are <{http://marklogic.com/manage}role-properties>
In the case of a 403 Forbidden error, with a message such as "You do not have credentials to access /manage/v2/databases/databaseName/properties", the message may be misleading in case the MarkLogic user you're using has granular privileges that allow the user to access the "databaseName" database, but not modify every particular property. The Manage API will return the same error message regardless of whether the user cannot access the resource or can access it but not modify a particular property.
So in this scenario, check the contents of your resource file to see if there are properties that your user is not allowed to modify. A good test is to remove nearly everything from the resource file except for its name - database-name in this case - and then try deploying again. If that succeeds, then your user is able to access the resource, and your resource file then has one or more properties that the user is not allowed to modify.
Deploying an application to MarkLogic generally involves two steps - deploying resources (databases, servers, users, etc), and loading modules into a modules database. Deploying resources happens via the Manage API on port 8002, while loading modules happens via a combination of two ports - 8000 for non-REST modules (called "asset" modules in the REST API docs), and an application-specific port for REST modules (options, services, and transforms).
If your deployment fails, you'll want to know if it's because of an error with deploying a resource or loading a module. Info-level logging will tell you this fairly quickly. If we have e.g. a bad REST module, like an invalid options file, we'll get logging like this:
gradle -i mlDeploy
... omitting some gradle logging ...
Execution failed for task ':mlDeployApp'.
> Error occurred while loading REST modules: Local message: /config/query write failed: Bad Request. Server Message: RESTAPI-INVALIDCONTENT: (err:FOER0000) Invalid content: Unexpected Payload...
For more information, please see Debugging module loading.
If you get an error pertaining to a connection issue while loading modules, please see Debugging module loading instead, which provides some help on debugging the connection.
Ignoring module loading, any connection issue would be to the Manage API port, which is 8002 by default.
To figure out what's going wrong, you'll of course want to start with info-level logging first. Then look at the list of properties that are printed when the task runs, e.g.
Initializing ml-gradle
Admin interface host: localhost
Admin interface username: admin
App name: example
App host: localhost
App REST port: 8123
REST admin username: admin
App Services username: admin
Manage host: localhost
Manage username: admin
Manage user with security role: admin
The above properties are based on a gradle.properties file containing the following:
mlHost=localhost
mlAppName=example
mlRestPort=8123
mlUsername=admin
mlPassword=(some password)
See Configuring security for more information about the different users that can be configured to talk to MarkLogic.
Info-level logging should also show you what the actual problem is. For example, if we're using the wrong password for the admin user, we'll get logging like this:
gradle -i mlDeploy
... omitting some Gradle logging ...
Executing command [com.marklogic.appdeployer.command.databases.DeployContentDatabasesCommand] with sort order [120]
Merging JSON files at locations: [src/main/ml-config/databases/content-database.json]
Checking for existence of resource: example-content
Sending XML GET request as user 'admin' to path: /manage/v2/databases
Logging HTTP response body to assist with debugging: {"errorResponse": {"statusCode":401,
"status":"Unauthorized",
"message":"401 Unauthorized"
}
}
... omitting more Gradle logging ...
Execution failed for task ':mlDeploy'.
> 401 Unauthorized
The command logging shows what user is being used to connect, and the host and port were logged at the start of the task. The 401 tells us that our user isn't authorized, which is a good hint to check the username/password properties.
See Debugging DMSDK with Gradle for more information.