Configuration

MapD Core Database has minimal configuration requirements with a number of additional configuration options. This topic describes the required and optional configuration changes you can use in your MapD Core Database instance.

Data Directory

Before starting MapD Core Database, you must initialize the persistent data directory. To do so, create an empty directory at the desired path, such as /var/lib/mapd. Create the environment variable $MAPD_STORAGE.

export MAPD_STORAGE=/var/lib/mapd

Change the owner of the directory to the user that the server will run as ($MAPD_USER):

sudo mkdir -p $MAPD_STORAGE
sudo chown -R $MAPD_USER $MAPD_STORAGE

Where $MAPD_USER is the system user account that the server runs as, such as mapd, and $MAPD_STORAGE is the path to the parent of the MapD Core Database data directory.

Finally, run $MAPD_PATH/bin/initdb with the data directory path as the argument:

$MAPD_PATH/bin/initdb $MAPD_STORAGE

Configuration file

MapD Core supports storing options in a configuration file. This is useful if, for example, you need to run the MapD Core Server and Web Server on ports different than the defaults.

If you store a copy of mapd.conf in the $MAPD_STORAGE directory, the configuration settings are picked up automatically by the sudo systemctl start mapd_server and sudo systemctl start mapd_web_server commands.

Set the flags in the configuration file using the format <flag> = <value>. Strings must be enclosed in quotes. The following is a sample configuration file. The entry for data path is a string and must be in quotes. The entry for the optional read-only flag is the Boolean value true and is not in quotes.

port = 9091
http-port = 9090
data = "/var/lib/mapd/data"
read-only = true


[web]
port = 9092
frontend =
    "/home/osboxes/installs/mapd-3.0.0-20170502-9e5ba95-Linux-x86_64-render/frontend"

Configuration Flags for MapD Server

Configuration Flags for MapD Server
Flag Description Default Why Change It?
cpu Set this flag on a GPU installation to instruct it to use only CPUs. (You do not have to explicitly set this flag for a CPU-only installation.) FALSE One use case for disabling GPUs is during database conversion, which requires moving a large amount of data with minimal processing.
gpu Run on GPUs and CPUs. TRUE Default.
read-only Enable read-only mode. FALSE Prevents inadvertent (or nefarious) changes to the dataset.
port Port number 9091 Change to avoid collisions with other services.
calcite-port Calcite port number 9093 Change to avoid collisions with other services.
ldap-uri LDAP server URI N/A N/A
ldap-ou-dc LDAP Organizational Unit and Domain Component =ou=users,dc=mapd,dc=com N/A
http-port HTTP port number 9090 Change the port number if it collides with another service on the host. Ideally, your host only runs MapD services.
flush-log Force aggressive log file flushes FALSE When you set this, the system writes messages to disk as they are generated, instead of holding them until a particular threshold is reached.
num-gpus Number of GPUs to use -1 In a shared environment, you can assign the number of GPUs to a particular application. The default is -1, which means use all available GPUs.
start-gpu First GPU to use 0 In a shared environment, if you want to reserve a set number of GPUs for a particular process, you can configure another process to use GPUs starting at a higher device ID.
cluster Indicates that the MapD Core Database instance is an aggregator node, and where to find the rest of its cluster $MAPD_STORAGE This setting is not likely to change in a production environment.
string-servers Path to string servers list JSON file $MAPD_STORAGE This setting is not likely to change in a production environment.

Configuration Flags for MapD Web Server

Configuration Flags for MapD Web Server
Flag Description Default Why Change It?
-b | backend-url URL to http-port on mapd_server http://localhost:9090 Change to avoid collisions with other services.
--cert Certificate file for HTTPS cert.pem Change for testing and debugging.
-c | --config Path to MapD configuration file   Change for testing and debugging.
-d | --data Path to MapD data directory data Change for testing and debugging.
--docs Path to documentation directory docs  
--enable-https Enable HTTPS support   Change to enable secure HTTP.
-f | --frontend Path to frontend directory frontend  
--key Key file for HTTPS key.pem Change for testing and debugging.
-p | --port Frontend server port 9092 Change to avoid collisions with other services.
-r | --read-only Enable read-only mode   Prevent inadvertent (or nefarious) changes to the data.
--servers-json Path to servers.json   Change for testing and debugging.
--timeout Maximum request duration in #h#m#s format 1h0m0s The --timeout option controls the maximum duration of individual HTTP requests. This is used to manage resource exhaustion caused by improperly closed connections. One side effect is that it limits the execution time of queries made over the Thrift HTTP transport. This timeout duration must be increased if queries are expected to take longer than the default duration of one hour; for example, if you perform a COPY FROM on a large file when using mapdql with the HTTP transport.
--tmpdir Path for temporary file storage /tmp The temporary directory is used as a staging location for file uploads. You may want to locate this directory on the same file system as the MapD Core data directory. If not specified on the command line, mapd_web_server also respects the standard TMPDIR environment variable as well as a specific MAPD_TMPDIR environment variable, the latter of which takes precedence. If you use neither the command-line argument nor one of the environment variables, the default, /tmp/, is used.
-v | --verbose Print all log messages to stdout   Change for testing and debugging.
--version Return version    

Using Configuration Flags on the Command Line

To use options provided in a configuration file, set the --config flag to the path of the configuration file for mapd_server and mapd_web_server. For example:

$MAPD_PATH/bin/mapd_server --config $MAPD_STORAGE/mapd.conf

You can also specify configuration settings at the command line. MapD recommends that you use the systemctl command to start and stop the servers, but you can use the mapd_server and mapd_web_server commands with configuration flags for testing and debugging.

Command Line Configuration Flags for mapd_server

Command Line Configuration Flags for mapd_server
Flag Description Default Why Change It?
--config arg Path to mapd.conf none You might want to set a different configuration file temporarily during testing and troubleshooting.
--data arg Directory path to MapD catalogs $PWD/data You can set the path anywhere.
--cpu Set this flag on a GPU installation to instruct it to use only CPUs. (You do not have to explicitly set this flag for a CPU-only installation.) FALSE You may want to disable GPUs during database conversion, which requires moving a large amount of data with minimal processing.
--gpu Run on GPUs and CPUs. TRUE Default.
--read-only Enable read-only mode. FALSE Prevents inadvertent (or nefarious) changes to the dataset.
-p [ --port ] arg Port number 9091 Change to avoid collisions with other services.
--calcite-port arg Calcite port number 9093 Change to avoid collisions with other services.
--ldap-uri arg LDAP server URI N/A N/A
--ldap-ou-dc arg LDAP Organizational Unit and Domain Component =ou=users,dc=mapd,dc=com N/A
--http-port arg HTTP port number 9090 Change the port number if it collides with another service on the host. Ideally, your host only runs MapD services.
--flush-log Force aggressive log file flushes FALSE When you set this, the system writes messages to disk as they are generated, instead of holding them until a particular threshold is reached.
--num-gpus arg Number of GPUs to use -1 In a shared environment, you can assign the number of GPUs to a particular application. The default is -1, which means use all available GPUs.
--start-gpu arg First GPU to use 0 In a shared environment, if you want to reserve a set number of GPUs for a particular process, you can configure another process to use GPUs starting at a higher device ID.
-v [ --version ] Print release version number. N/A N/A
--cluster arg Indicates that the MapD Core Database instance is an aggregator node, and where to find the rest of its cluster $MAPD_STORAGE This setting is not likely to change in a production environment.
--string-servers arg Path to string servers list JSON file $MAPD_STORAGE This setting is not likely to change in a production environment.

Command Line Configuration Flags for mapd_web_server

Command Line Configuration Flags for mapd_web_server
Flag Description Default Why Change It?
-b | backend-url string URL to http-port on mapd_server http://localhost:9090 Change to avoid collisions with other services.
--cert string Certificate file for HTTPS cert.pem Change for testing and debugging.
-c | --config string Path to MapD configuration file   Change for testing and debugging.
-d | --data string Path to MapD data directory data Change for testing and debugging.
--docs string Path to documentation directory docs  
--enable-https Enable HTTPS support   Change to enable secure HTTP.
-f | --frontend string Path to frontend directory frontend  
--key string Key file for HTTPS key.pem Change for testing and debugging.
-p | --port int Frontend server port 9092 Change to avoid collisions with other services.
-r | --read-only Enable read-only mode   Prevent inadvertent (or nefarious) changes to the data.
--servers-json string Path to servers.json   Change for testing and debugging.
--timeout duration Maximum request duration in #h#m#s format. For example 0h30m0s represents a duration of 30 minutes. 1h0m0s The --timeout option controls the maximum duration of individual HTTP requests. This is used to manage resource exhaustion caused by improperly closed connections. One side effect is that it limits the execution time of queries made over the Thrift HTTP transport. This timeout duration must be increased if queries are expected to take longer than the default duration of one hour; for example, if you perform a COPY FROM on a large file when using mapdql with the HTTP transport.
--tmpdir string Path for temporary file storage /tmp The temporary directory is used as a staging location for file uploads. You may want to place this directory on the same file system as the MapD Core data directory. If not specified on the command line, mapd_web_server also respects the standard TMPDIR environment variable as well as a specific MAPD_TMPDIR environment variable, the latter of which takes precedence. If you use neither the command-line argument nor one of the environment variables, the default, /tmp/ is used.
-v | --verbose Print all log messages to stdout   Change for testing and debugging.
--version Return version