Vjylku

SFDX - Create Objects with Custom Properties

Can a level 2 Warlock take one level in rogue, then continue advancing as a warlock?

Why is the underscore command _ useful?

Is there any pythonic way to find average of specific tuple elements in array?

A strange hotel

Is there a word for the censored part of a video?

What *exactly* is electrical current, voltage, and resistance?

How to have a sharp product image?

Restricting the options of a lookup field, based on the value of another lookup field?

How can I get rid of an unhelpful parallel branch when unpivoting a single row?

Drawing a german abacus as in the books of Adam Ries

How can I practically buy stocks?

Work requires me to come in early to start computer but wont let me clock in to get paid for it

How much cash can I safely carry into the USA and avoid civil forfeiture?

Is Electric Central Heating worth it if using Solar Panels?

How long after the last departure shall the airport stay open for an emergency return?

"The cow" OR "a cow" OR "cows" in this context

What makes accurate emulation of old systems a difficult task?

Extracting Dirichlet series coefficients

Unknown code in script

Why did Rep. Omar conclude her criticism of US troops with the phrase "NotTodaySatan"?

Where was the County of Thurn und Taxis located?

Contradiction proof for inequality of P and NP?

"My boss was furious with me and I have been fired" vs. "My boss was furious with me and I was fired"

How to prevent GDAL-dissolve from ignoring attribute encoding?

Shapefile encoding problem in QGIS 1.9.0 (built with GDAL 1.9.2)Connecting to PostGIS (LATIN1) from Microsoft Access 2010 - Encoding problem?How to dissolve naturalearth vectordata with gdal?dissolve polygons of GeoJSON with gdal/ogrDissolve using GDAL/OGR?PostGIS dissolve geometries from shapefilesEncoding error 2006 from Carto?Is there an equivalent to the Dissolve all function in GDAL dissolve user interface?How to extract the text encoding from a shapefile?Dissolve not based on attribute in geopandas

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}

0

Given a huge shapefile which needs desolving by one attribute field to reduce redundancy. The GDAL dissolve functionality does that quite swiftly but the output attributes change the encoding so äöü become to unusable ?squares.

What i've tried so far:

• Adding a hugeshpfile.cpg that additionaly declares UTF-8: ?squares


• Doing the detour via Postgis: same result


• The native QGIS dissolve function freezes no matter if shapefile or postgis


• Doing it via SQL query in pgadmin results in a different error messages.

While trying around i didn't encounter a possibility to influence the encoding. How can that be achieved or am i missing out on some format constraints?

postgis gdal dissolve encoding utf-8

share|improve this question

asked Mar 4 '17 at 23:59

maxwheremaxwhere

324212

bumped to the homepage by Community♦ 17 mins ago

This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I'm not sure what the detour in postgis means. Hopefully not the same as this and it helps you out - postgis.net/docs/ST_Union.html
  
  – user92055
  Mar 5 '17 at 18:36
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Could you create a small dataset for testing? Two or three features with an attribute to use for dissolve and containing äöü should be enough.
  
  – user30184
  Mar 5 '17 at 19:45
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @user30184: Testdataset 35-3_35-4_merged.shp is a part of the merged subset. *_dissolved is from native qgis with correct encoding; *_diss_gdal is made with ST_Union
  
  – maxwhere
  Mar 5 '17 at 20:17
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  What shapefile presents the original data? What attribute should I use for dissolving? Is it 35-3_35-4_merged.shp and attribute "LAYER"? Or have I even understood right what you try to do?
  
  – user30184
  Mar 5 '17 at 20:42
  
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  The original data comes in tiles of 2,5km². 35-3_35-4_merged.shp represents two of those and i wand to dissolve them by the field 'FMZK_ID'. If i do this with qgis native function it takes forever, if i do it with gdal disssolve polygons it destroys the encoding.
  
  – maxwhere
  Mar 6 '17 at 13:07
  

  
  
  
  

 | 
show 1 more comment

0

Given a huge shapefile which needs desolving by one attribute field to reduce redundancy. The GDAL dissolve functionality does that quite swiftly but the output attributes change the encoding so äöü become to unusable ?squares.

What i've tried so far:

• Adding a hugeshpfile.cpg that additionaly declares UTF-8: ?squares


• Doing the detour via Postgis: same result


• The native QGIS dissolve function freezes no matter if shapefile or postgis


• Doing it via SQL query in pgadmin results in a different error messages.

While trying around i didn't encounter a possibility to influence the encoding. How can that be achieved or am i missing out on some format constraints?

postgis gdal dissolve encoding utf-8

share|improve this question

asked Mar 4 '17 at 23:59

maxwheremaxwhere

324212

bumped to the homepage by Community♦ 17 mins ago

This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I'm not sure what the detour in postgis means. Hopefully not the same as this and it helps you out - postgis.net/docs/ST_Union.html
  
  – user92055
  Mar 5 '17 at 18:36
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Could you create a small dataset for testing? Two or three features with an attribute to use for dissolve and containing äöü should be enough.
  
  – user30184
  Mar 5 '17 at 19:45
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @user30184: Testdataset 35-3_35-4_merged.shp is a part of the merged subset. *_dissolved is from native qgis with correct encoding; *_diss_gdal is made with ST_Union
  
  – maxwhere
  Mar 5 '17 at 20:17
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  What shapefile presents the original data? What attribute should I use for dissolving? Is it 35-3_35-4_merged.shp and attribute "LAYER"? Or have I even understood right what you try to do?
  
  – user30184
  Mar 5 '17 at 20:42
  
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  The original data comes in tiles of 2,5km². 35-3_35-4_merged.shp represents two of those and i wand to dissolve them by the field 'FMZK_ID'. If i do this with qgis native function it takes forever, if i do it with gdal disssolve polygons it destroys the encoding.
  
  – maxwhere
  Mar 6 '17 at 13:07
  

  
  
  
  

 | 
show 1 more comment

Given a huge shapefile which needs desolving by one attribute field to reduce redundancy. The GDAL dissolve functionality does that quite swiftly but the output attributes change the encoding so äöü become to unusable ?squares.

What i've tried so far:

• Adding a hugeshpfile.cpg that additionaly declares UTF-8: ?squares


• Doing the detour via Postgis: same result


• The native QGIS dissolve function freezes no matter if shapefile or postgis


• Doing it via SQL query in pgadmin results in a different error messages.

While trying around i didn't encounter a possibility to influence the encoding. How can that be achieved or am i missing out on some format constraints?

postgis gdal dissolve encoding utf-8

share|improve this question

asked Mar 4 '17 at 23:59

maxwheremaxwhere

324212

share|improve this question

asked Mar 4 '17 at 23:59

maxwheremaxwhere

324212

share|improve this question

asked Mar 4 '17 at 23:59

maxwheremaxwhere

324212

asked Mar 4 '17 at 23:59

maxwheremaxwhere

324212

asked Mar 4 '17 at 23:59

maxwheremaxwhere

324212

1 Answer
1

active

oldest

votes

0

I solved my problem by avoiding it with a workaround that replaces all non-ascii characters in the affected fields with their compatible equivalents by using a script i derived from this answer:

def encode(shapefile):

    d = {'ä':'ae','ü':'ue','ö':'oe','ß':'ss',

    'Ä':'Ae','Ü':'Ue','Ö':'Oe'}

    driver = ogr.GetDriverByName("ESRI Shapefile")

    dataSource = driver.Open(shapefile, 1)

    layer = dataSource.GetLayer()

    for feature in layer:

         field = feature.GetField("LAYER")

         pattern = re.compile("([" + "".join(d.keys()) + "]).")

         repl = pattern.sub(lambda x: d[x.group()], field)

         feature.SetField("LAYER", repl)

         layer.SetFeature(feature)

    layer.ResetReading()

Surely this function could be better designed and although it does the job I'm still open to suggestions concerning the encoding since this workaround shouldn't have been necessary in the first place.

share|improve this answer

edited May 23 '17 at 12:39

Community♦

1

answered Mar 7 '17 at 23:55

maxwheremaxwhere

324212

add a comment | 

Your Answer

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "79"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fgis.stackexchange.com%2fquestions%2f230837%2fhow-to-prevent-gdal-dissolve-from-ignoring-attribute-encoding%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

0

I solved my problem by avoiding it with a workaround that replaces all non-ascii characters in the affected fields with their compatible equivalents by using a script i derived from this answer:

def encode(shapefile):

    d = {'ä':'ae','ü':'ue','ö':'oe','ß':'ss',

    'Ä':'Ae','Ü':'Ue','Ö':'Oe'}

    driver = ogr.GetDriverByName("ESRI Shapefile")

    dataSource = driver.Open(shapefile, 1)

    layer = dataSource.GetLayer()

    for feature in layer:

         field = feature.GetField("LAYER")

         pattern = re.compile("([" + "".join(d.keys()) + "]).")

         repl = pattern.sub(lambda x: d[x.group()], field)

         feature.SetField("LAYER", repl)

         layer.SetFeature(feature)

    layer.ResetReading()

Surely this function could be better designed and although it does the job I'm still open to suggestions concerning the encoding since this workaround shouldn't have been necessary in the first place.

share|improve this answer

edited May 23 '17 at 12:39

Community♦

1

answered Mar 7 '17 at 23:55

maxwheremaxwhere

324212

add a comment | 

0

I solved my problem by avoiding it with a workaround that replaces all non-ascii characters in the affected fields with their compatible equivalents by using a script i derived from this answer:

def encode(shapefile):

    d = {'ä':'ae','ü':'ue','ö':'oe','ß':'ss',

    'Ä':'Ae','Ü':'Ue','Ö':'Oe'}

    driver = ogr.GetDriverByName("ESRI Shapefile")

    dataSource = driver.Open(shapefile, 1)

    layer = dataSource.GetLayer()

    for feature in layer:

         field = feature.GetField("LAYER")

         pattern = re.compile("([" + "".join(d.keys()) + "]).")

         repl = pattern.sub(lambda x: d[x.group()], field)

         feature.SetField("LAYER", repl)

         layer.SetFeature(feature)

    layer.ResetReading()

Surely this function could be better designed and although it does the job I'm still open to suggestions concerning the encoding since this workaround shouldn't have been necessary in the first place.

share|improve this answer

edited May 23 '17 at 12:39

Community♦

1

answered Mar 7 '17 at 23:55

maxwheremaxwhere

324212

add a comment | 

I solved my problem by avoiding it with a workaround that replaces all non-ascii characters in the affected fields with their compatible equivalents by using a script i derived from this answer:

def encode(shapefile):

    d = {'ä':'ae','ü':'ue','ö':'oe','ß':'ss',

    'Ä':'Ae','Ü':'Ue','Ö':'Oe'}

    driver = ogr.GetDriverByName("ESRI Shapefile")

    dataSource = driver.Open(shapefile, 1)

    layer = dataSource.GetLayer()

    for feature in layer:

         field = feature.GetField("LAYER")

         pattern = re.compile("([" + "".join(d.keys()) + "]).")

         repl = pattern.sub(lambda x: d[x.group()], field)

         feature.SetField("LAYER", repl)

         layer.SetFeature(feature)

    layer.ResetReading()

Surely this function could be better designed and although it does the job I'm still open to suggestions concerning the encoding since this workaround shouldn't have been necessary in the first place.

share|improve this answer

edited May 23 '17 at 12:39

Community♦

1

answered Mar 7 '17 at 23:55

maxwheremaxwhere

324212

def encode(shapefile):

    d = {'ä':'ae','ü':'ue','ö':'oe','ß':'ss',

    'Ä':'Ae','Ü':'Ue','Ö':'Oe'}

    driver = ogr.GetDriverByName("ESRI Shapefile")

    dataSource = driver.Open(shapefile, 1)

    layer = dataSource.GetLayer()

    for feature in layer:

         field = feature.GetField("LAYER")

         pattern = re.compile("([" + "".join(d.keys()) + "]).")

         repl = pattern.sub(lambda x: d[x.group()], field)

         feature.SetField("LAYER", repl)

         layer.SetFeature(feature)

    layer.ResetReading()

share|improve this answer

edited May 23 '17 at 12:39

Community♦

1

answered Mar 7 '17 at 23:55

maxwheremaxwhere

324212

edited May 23 '17 at 12:39

Community♦

1

edited May 23 '17 at 12:39

Community♦

1

answered Mar 7 '17 at 23:55

maxwheremaxwhere

324212

answered Mar 7 '17 at 23:55

maxwheremaxwhere

324212