Vjylku

I am building a stand alone arcpy script, and need to connect to the specific feature classes within a published web map, query/copy from those feature classes, then after some processing update those specific feature classes with new or modified data.

I have a GUID value to the specific web map I need to interact with (this is my only point of entry to the underlying feature classes), but I need to be able to find the database connections to the database and feature classes that make up the web map. The web map was constructed from 3 published web feature layers/feature services, each containing 1-many feature classes, and I ultimately need to obtain the database connection to a specific point feature and line feature, run my script (which makes copies of those feature classes then runs arcpy geo-processing steps-near_analysis, snap, SplitLineAtPoint, AddGeometryAttributes) then need to update/add features back to the respective database and feature classes.

My querying/geo-processing script is already working on hardcoded database connections, but now I need to loop through the various GUIDs I have to different web maps and dynamically connect to variable database connections within the web map (that I have a GUID ID for).

I have found documentation showing how to access webmaps:

from arcgis.gis import GIS

gis = GIS()

web_map = gis.content.get('<GUID value>')#where GUID value is the GUID I have in a list

and then accessing the web feature layers that make up the web map:

web_map_layers = webMap(web_map)

layers = web_map_layers.layers

but printing that out just seems to give me a list of dictionaries, not something that I can assign to a variable and treat as a real feature class within a database or even temporarily in memory.

I was finally able to get this to work, by taking the WebMap guid ID supplied, and:

1. find out the Feature Service of the feature I wanted to copy


2. hit the REST end point of that feature service


3. copy that feature set down to a feature class.

This then allowed me to run my geo-processing tools, and report my findings. The only thing I have yet to figure out is how to push my updates/additions back to the underlying feature classes in the feature services...

Here is the code that worked for me (with some data/code redacted/changed):

import arcpy

from arcpy import env

import timeit

import uuid

from arcgis.gis import GIS

from arcgis import features

from arcgis.mapping import WebMap

from collections import OrderedDict

import json

import urllib

import http.client



start_time = timeit.default_timer()



#Constant values

SURVEY_DB_WORKSPACE = r'C:Userssurvey_gdb.sde'

#survey table server instance and owner prefix values

SURVEY_DB_INSTANCE_AND_OWNER = 'mydb.dbo.'

GEO_PROC_WORKSPACE = r"C:Usersgeo_proc_gdb.sde"

#geo-proc server instance and owner prefix values

GEO_PROC_DB_INSTANCE_AND_OWNER = 'mydb_gdb.DBO.'

#rest services credentials

USERNAME = 'user'

PASSWORD = 'password'

PORTAL_URL = 'https://mywebsite/portal'

SERVERNAME = "mywebsite"

PORT = ''#if using web adaptor, leave empty (modify tokenURL)

TOKEN_URL = "https://mywebsite/server/admin/generateToken"



def main():



    #set environment to the Survey database, loop through surveys table, grab WebMap GUID

    workspace = SURVEY_DB_WORKSPACE

    arcpy.env.workspace = workspace

    arcpy.env.overwriteOutput = True



    surveys = SURVEY_DB_INSTANCE_AND_OWNER + 'Surveys'

    where_clause = "active = 1"

    fields = ['OID@', 'Name', 'Id', 'Progress']

    web_map_ids = OrderedDict()



    with arcpy.da.SearchCursor(surveys, fields) as cursor:

        for row in cursor:

            web_map_ids[row[0]] = [row[1], row[2], row[3]]



    #change environment to the geo-processing database

    workspace = GEO_PROC_WORKSPACE



    #connect to db and set point and line features

    field = 'state'

    lines_backup = GEO_PROC_DB_INSTANCE_AND_OWNER + 'lines_backup'

    points_backup = GEO_PROC_DB_INSTANCE_AND_OWNER + 'points_backup'

    tmp_lines_split = GEO_PROC_DB_INSTANCE_AND_OWNER + 'tmp_lines_split'

    lines_split = GEO_PROC_DB_INSTANCE_AND_OWNER + 'lines_split'

    arcpy.env.workspace = workspace

    arcpy.env.overwriteOutput = True



    #Define how to acquire a rest services token

    #from https://community.esri.com/thread/83654



    def getToken(username, password, serverName, serverPort):

        params = urllib.parse.urlencode({'username': USERNAME, 'password': PASSWORD,'client': 'requestip', 'f': 'json'})

        headers = {"Content-type": "application/x-www-form-urlencoded", "Accept": "text/plain"}



        # Connect to URL and post parameters

        httpConn = http.client.HTTPSConnection(serverName, serverPort)

        httpConn.request("POST", TOKEN_URL, params, headers)



        # Read response

        response = httpConn.getresponse()

        print(response.status)

        if (response.status != 200):

            httpConn.close()

            print("Error while fetching tokens from admin URL. Please check the URL and try again.")

            return

        else:

            data = response.read()

            httpConn.close()



        # Check that data returned is not an error object

        if not assertJsonSuccess(data): 

            return



        # Extract the token from it

        token = json.loads(data) 

        return token['token']



        ###End getToken function



# A function that checks that the input JSON object is not an error object. 

def assertJsonSuccess(data):

    obj = json.loads(data)

    if 'status' in obj and obj['status'] == "error":

        print("Error: JSON object returns an error. " + str(obj))

        return False

    else:

        return True



    ### End jsonSuccess function

    """

        This is the Main loop through WebMap values, processing and reporting back to survey, then continuing

    """

    #loop through the survey web map guids for processing-this is where the GUID values come from

    current_survey = ''

    for k,v in web_map_ids.items():

        if v[1] is None:

            continue

        current_survey = v[0]



        #get WebMap

        gis = GIS(PORTAL_URL, username=USERNAME, password=PASSWORD)

        web_map = gis.content.get(v[1])

        web_Map = WebMap(web_map)

        web_map_layers = web_Map.layers



        #extract the feature service name that contains the Point feature class

        num = 0

        for item in web_map_layers:

            for tag in item:

                if tag =='title':

                    if item[tag] == 'Point':

                        num = web_map_layers.index(item)

                    else:

                        continue

                else:

                    continue



        point_url = web_map_layers[num]['url']



        #from https://www.esri.com/arcgis-blog/products/arcgis-desktop/analytics/quick-tips-consuming-feature-services-with-geoprocessing/

        #Extract Points from feature service to feature class

        baseURL = point_url

        where = '1=1'

        fields = '*'

        token = 'getToken(USERNAME, PASSWORD, SERVERNAME, PORT)'

        query = "?where={}&outFields={}&returnGeometry=true&f=json&token={}".format(where, fields, token)

        # See http://services1.arcgis.com/help/index.html?fsQuery.html for more info on FS-Query

        fsURL = baseURL + query

        fs = arcpy.FeatureSet()

        fs.load(fsURL)



        #clean up prior to re-creating backups

        if arcpy.Exists(lines_backup):

            arcpy.Delete_management(lines_backup)

        if arcpy.Exists(points_backup):

            arcpy.Delete_management(points_backup)



        #copy Points from feature service to feature class, for geo-processing

        arcpy.CopyFeatures_management(fs, points_backup)

Once I was able to copy the feature set from the feature service to a feature class, I was able to then run the rest of my geo-processing script. I have not yet been able to push updates/additions back up to those feature classes underlying the feature services though. tackling that now...