Data Loading Tools

Work with data loading workflows

The scenarios below describe various situations for loading data when mapping a single dataset or multiple datasets. Review the options below to see what workflow best fits your needs.

Append with Transformation tool

This tool is best suited for an ad hoc data loading process or processing single datasets. To begin, input a dataset, use a Python expression to transform the dataset, and view the result.

  1. Start ArcGIS Pro and select an existing project, create a project, or choose Start without a template.
  2. On the Analysis tab, in the Geoprocessing group, click Tools Tools.
  3. In the Geoprocessing pane, click the Toolboxes tab to view the toolboxes provided with ArcGIS Pro.
  4. Browse to the Data Loading Tools toolbox and click the drop-down arrow to view the included tools.
  5. Click Append with Transformation to open the tool.
  6. Update the tool with the following parameter information:
    1. Specify the Source data parameter with the location of the source dataset.
    2. Specify the Target data parameter with the location of the target dataset.
    3. Specify the Target Field and Expression parameters under the Field Mapping section.
    4. Optionally, use the Code Block parameter for multiline Python expressions and call it in the Expression parameter to run.
    5. Optionally, select Preview for the Load Data To: parameter to load data into a preview geodatabase instead of your target workspace. Then specify an output folder for the Preview Output Folder parameter.
  7. Click Run.

Learn more about the Append with Transformation tool.

Data Loading Workspace

If you have multiple datasets that you need to load into a new dataset, the following workflow describes the steps to accomplish this goal. This option creates a set of workbooks that can be iterated as your process evolves. It also allows you to have a record of your mapping information so that it can be fine-tuned or revisited as needed.

Create a workspace

First, create a Data Loading Workspace.

  1. Start ArcGIS Pro and select an existing project, create a project, or choose Start without a template.
  2. On the Analysis tab, in the Geoprocessing group, click Tools Tools.
  3. In the Geoprocessing pane, click the Toolboxes tab to view the toolboxes provided with ArcGIS Pro.
  4. Browse to the Data Loading Tools toolbox and click the drop-down arrow to view the included tools.
  5. Click Create Data Loading Workspace to open the tool.
  6. Update the tool with the following parameters:
    1. Specify the Source to Target Mapping parameter with the location of the source dataset and the corresponding target dataset.
    2. Optionally, enable the Predictive Field Matching Options parameters. The options include the following:

      • Field Name Similarity—This option matches field names based on similarity between the source and target.
      • Attribute Domain Coded Value Description Similarity—This option matches attribute domain value descriptions based on similarity between the source and target.

    3. Optionally, specify a table for the Predefined Mapping parameter or create a table by clicking the create table button.

    4. Specify the name of the data loading workspace in the Output Folder parameter.
  7. Click Run.
    This tool creates a Data Loading Workspace with a set of workbooks for data loading.

Map the source to the target

The Create Data Loading Workspace tool generates a folder with a series of Microsoft Excel workbooks for performing data mapping. In this workspace, there are two types of information: resource and mapping. The resource files are generated to assist in the mapping of your data and include domain workbooks and sample scripts. The mapping files include the Data Reference and the Data Mapping workbooks, where the majority of your mapping work will be performed. How you map your data depends on your source and target datasets. Before mapping your data, review Data Loading Workspace for an overview of the components and how they function.

Load the data

Once the workbooks have been generated and the mapping workbooks filled in, you can load the data and review how the data was carried over from the source to the target schema. Depending on the quality or complexity of the data, it is recommended that you work with a subset of the data and iterate through the mapping and loading process as needed.

  1. Start ArcGIS Pro and select an existing project, create a project, or choose Start without a template.
  2. On the Analysis tab, in the Geoprocessing group, click Tools Tools.
  3. In the Geoprocessing pane, click the Toolboxes tab to view the toolboxes provided with ArcGIS Pro.
  4. Browse to the Data Loading Tools toolbox and click the drop-down arrow to view the included tools.
  5. Click Execute Data Load to open the tool.
  6. Update the tool parameters for Execute Data Load.

    • Choose the DataReference.xlsx workbook as the input for the Data Reference workbook parameter.
    • Optionally, select Preview for the Load Data To: parameter to load data into a preview geodatabase instead of your target workspace. Then specify an output folder for the Preview Output Folder parameter.

  7. Click Run.

    After loading your data, review the results and verify the quality of your data mapping. There may be gaps or missing information that was not properly captured, and you must go back to the mapping workbooks to edit your settings before rerunning the Execute Data Load tool. This is an iterative process, which is why it is recommended that you work with a subset of the data to start.

Update a data loading workspace schema

In this workflow, you have an existing data loading workspace that you have been using but your source/target workspace has changed in fields and/or domains. You now want to update the data loading workbooks to reflect these updates.

  1. Start ArcGIS Pro and select an existing project, create a project, or choose Start without a template.
  2. On the Analysis tab, in the Geoprocessing group, click Tools Tools.
  3. In the Geoprocessing pane, click the Toolboxes tab to view the toolboxes provided with ArcGIS Pro.
  4. Browse to the Data Loading Tools toolbox and click the drop-down arrow to view the included tools.
  5. Click Update a Data Loading Workspace Schema to open the tool.

Generate a predefined mapping table

Use an existing Data Loading Workspace to refine a future iteration of your data migration or to share with others to jumpstart their migration. Use the Generate Predefine Mapping Table tool to create a list of the mapped datasets, fields, and attribute domain coded value descriptions. Use the steps below to generate a Predefined Mapping table.

  1. Start ArcGIS Pro and select an existing project, create a project, or choose Start without a template.
  2. On the Analysis tab, in the Geoprocessing group, click Tools Tools.
  3. In the Geoprocessing pane, click the Toolboxes tab to view the toolboxes provided with ArcGIS Pro.
  4. Browse to the Data Loading Tools toolbox and click the drop-down arrow to view the included tools.
  5. Click Generate Predefined Mapping Table to open the tool.
  6. Update the tool parameters for Generate Predefined Mapping Table as follows:

    • Choose the DataReference.xlsx workbook as the input for the Data Reference workbook parameter.
    • Use the default settings for the Output Table parameter.

The Predefined Mapping table can be used in the Create Data Loading Workspace tool. When creating a Data Loading Workspace, specify this table for the Predefined Mapping parameter.

To learn more, see Data Loading Workspace.

Top