Geoprocessing Service - Create SD File about hydrovis HOT 4 OPEN

Note: Bob did find some documentation on how to automate the publishing of a geoprocessing tool... but it looks a bit cumbersome... so like the print tool, manual publishing may make sense for now.

Initial geoprocessing tool package sent to Bob on 0309. Will coordinate with him next week on publishing to TI EGIS, testing, and developing a plan to publish the tool to the other EGIS environments.

from hydrovis.

Update: I finally got Arc Pro to republish the test geoprocessing service, and I simplified the input arguments to be super simple (albeit there are quite a few of them). This allowed me to test again, to see if Bob's attempts to install boto3 worked, per this documentation: https://enterprise.arcgis.com/en/server/latest/publish-services/windows/deploying-custom-python-packages.htm

Unfortunately, I'm still getting the error that boto3 module cannot be found, so it seems that didn't work. I've asked Bob to look into this again.

This can be tested by submitting a geoprocessing job with the default parameters here: https://maps-testing.water.noaa.gov/server/rest/services/test_demo/MapxToSD_test2/GPServer/MapxToSD_test2

from hydrovis.

Background
At this point, all services have been moved off of the Viz EC2 instance, and it only performs one task: converting ArcGIS Pro Projects (which we save as MapX documents) into the ArcGIS service definition (SD) files that viz pipelines need to automatically publish services to ArcGIS server, including updating the data sources with the correct database host and authentication information.

The Viz EC2 instance does this by running the \aws_loosa\deploy\update_data_stores_and_sd_files.py script as part of the Core\EC2\viz\scripts\viz_ec2_setup.ps1 deployment script, and this is the #1 cause of issues during all deployments, as it takes quite a bit of work to dig into the logs and troubleshoot issues on the Viz EC2 machine. Because ESRI is greedy and won't fully allow their Python API to update these map document parameters (or allow use of a limited version of ArcPy in a container that can do it), we've had to get creative on another way to automatically update these map service definitions as part of our automatic pipelines - credit to Justin for initially coming up with the idea of using a geoprocessing service.

With Bob's help, I've successfully tested this geoprocessing service in TI, and while questions remain around how this will actually work operationally with the regular pipelines, it seems worth a shot. That said, because it is ESRI, there is a pretty convoluted workflow required to make this happen.

Steps Required for Implementation (These should be setup as new Issues assigned to the appropriate parties)

• Convert the update_data_stores_and_sd_files python script into a standalone script that can be published as a ArcGIS Geoprcessing Service (in this PR).
• Test the geoprocessing service via REST - you can find the test here.
• Allow the ESRISSMDeploy IAM Role to view the EGIS RDS user secret (this allows the geoprocessing server to fetch the RDS password securely from AWS without the password being passed around in text in the REST calls). The Terraform for this is included in this PR.
• Configure ArcGIS Server to use a custom python environment with the AWS Boto3 library installed. Bob did this in TI following the instructions of this ESRI doc. I believe this will need to be baked into the AMI, so this task should be assigned to whoever owns that process. I'm not sure which server of the ArcGIS stack this should be installed on, probably the same one as the print service?
• Publish the geoprocessing service to each ArcGIS Server stack - This is unfortunately a manual process that must be done with the admin credentials. I'll include details in comment on this PR below.
• Update the viz_publish_service lambda function to call this geoprocessing service first via API when it needs to republish a service. This can be done now in TI, using the test geoprocessing service I created, to test this out with the operational TI pipelines.
• Figure out the best way to run pipelines / use this tool upon a new deployment, to ensure that updates services are re-published.
• Delete the Viz EC2 entirely (it should be manually stopped after a deployment for the time being, as it doesn't serve any purpose after the initial deployment steps).

Here are the steps that were required to publish the geoprocessing service to the EGIS in TI. This will need to be manually done on each EGIS instance, per the notes above. The noted custom python environment does not need to be configured on ArcGIS server in order to do these publishing steps (although the published geoprocessing service won't work until that is done). However, the HydrovisESRISSMDeploy AWS Role does need to have permissions to read the EGIS database secret in order to perform these steps, so that part of this PR will need to be deployed first (if you let the script get the egis db password from aws instead of providing the password directly to the script, which is recommended).

1. On an EC2 with ArcPro, copy the script in this PR to the computer, and include in that same directly a folder called files with Empty_Project.aprx in it (this is needed for the script, and can be found in the same utils folder of the repo with the script file). You will also need to sign-in to the Portal of the EGIS you're planning to publish to with the admin credentials.

2. In ArcCatalog, create a new toolbox (regular is fine, doesn't need to by Python Toolbox), and create a new script inside of that toolbox.
   

3. Set the name and set the Script File to point to the script contained in this PR, and check the box for Import script (no other options of the first page need to be checked). On the parameters page, setup parameters for all the input arguments listed in the script (make sure egis_db_password is optional, as providing the secret name is the more secure method). Click OK to create the script tool:
   

4. Open the new script tool, and setup a test run with a real service. Getting it to run successfully may take some troubleshooting.
   

5. Once it runs successfully, click Open History, right click on the successful run, go to Share As -> Share Web Tool (this will be greyed out if you're not logged in as the admin account). Assign the description and tags that you want, choose to copy data to the server, and choose the Server to publish to (I chose GP in TI, which makes sense... but this will need to be done in UAT and Prod in coordination with the EGIS folks, to ensure that it's the server they want running these workflows, and the same server the custom python environment is installed on). Don't share with everyone (e.g. public), only authenticated users:
   

When you validate, you'll probably get some warnings about Empty_Project.aprx and some connection string being uploaded to the server, that's fine and good. Once the tool is published, and the custom python environment with Boto3 is setup on the server, you should be able to test the tool through a REST job, as is possible in the test service I created here: https://maps-testing.water.noaa.gov/gp/rest/services/test_demo/MapxToSD_Take3/GPServer/MapxToSD/submitJob

from hydrovis.

Relevant Feature Branch: mapx_to_sd_geoprocessing_service

from hydrovis.