2

I'm trying to create a field in a vector layer and then update the values based on the sum of two fields which are already present in the table, using the Python console (QGIS3.4). Also I'm still learning Python... I originally started to develop this under 2.18, but I've recently had to move to QGIS 3 (which is part of the problem)- In 2.18 the first part of the script used to work i.e. creation of a new field in the table.

The original code: 

from qgis.core import *
from qgis.utils import *
from qgis.gui import *
from PyQt4.QtCore import QVariant
import processing
import os

layer_P = QgsProject.instance().mapLayersByName("BXO")[0]

SUM = QgsField('t_branch', QVariant.Int)
layer_P.addAttribute (SUM)
idx = layer_P.fieldNameIndex('t_branch')

layer_P.startEditing()

e = QgsExpression ('CB_Num_Ent' + 'CB_Num_Ext')
e.prepare( 't_branch'() )

for f in layer_P.getFeatures():
    f[idx] = e.evaluate( f )
    layer_P.updateFeature( f )

layer_P.updateField()
layer_P.commitChanges()

I've been looking at the API and seen that fieldNameIndex has been replaced with fields() .lookupField() or fields() .indexFromName() and since I've tried a couple of variations - for example:

layer_P = QgsProject.instance().mapLayersByName("BXO")[0]

SUM = QgsField('t_branch', QVariant.Int)
layer_P.addAttribute (SUM)

idx = layer_P.lookupField('t_branch')

layer_P.startEditing()

e = QgsExpression ('CB_Num_Ent' + 'CB_Num_Ext')
e.prepare( layer_P.fields() )

for f in layer_P.getFeatures():
    f[idx] = e.evaluate( f )
    layer_P.updateFeature( f )

layer_P.updateField()
layer_P.commitChanges()

but for the minute I'm still get the same type of error message ...AttributeError: 'QgsVectorLayer' object has no attribute 'lookupField'

So, firstly I'm trying to find out how to modify the script following the removal of indexFromName. And if that works, hopefully update my new field based on the calculation here : 

> e = QgsExpression ('CB_Num_Ent' + 'CB_Num_Ext')

For info the script was adapted from the following : Adding field and calculating expression with PyQGIS

PyQGIS: Multiply Fields and populate new field

How to fill fields with layer name in pyqgis

Vince
  • 20,017
  • 15
  • 45
  • 64
lhowarth
  • 551
  • 3
  • 14

2 Answers2

8

Please try the following code- this worked for me in the python console in QGIS 3.4.

In addition to layer.fieldNameIndex changing to layer.fields().lookupField or layer.fields().indexFromName, there has also been changes to the QgsExpression class. The evaluate() method which took a QgsFeature object as its argument in 2.X has been removed. It now takes a QgsExpressionContext() object. This context has a setFeature() method to which a QgsFeature is passed.

The prepare() method of QgsExpression which took a QgsFields argument in 2.X, now also takes a Context argument. To prepare the expression for evaluation, we need to create a QgsExpressionContextScope object, call setFields, passing the layer fields, then append the scope to the context.

from PyQt5.QtCore import QVariant

layer = QgsProject().instance().mapLayersByName('BXO')[0]
prov = layer.dataProvider()
fld = QgsField('t_branch', QVariant.Int)
prov.addAttributes([fld])
layer.updateFields()
idx = layer.fields().lookupField('t_branch')
layer.startEditing()


e = QgsExpression('CB_Num_Ent + CB_Num_Ext')
c = QgsExpressionContext()
s = QgsExpressionContextScope()
s.setFields(layer.fields())
c.appendScope(s)
e.prepare(c)


for f in layer.getFeatures():
    c.setFeature(f)
    value = e.evaluate(c)
    atts = {idx: value}
    layer.changeAttributeValues(f.id(), atts)
layer.commitChanges()

EDIT: I found that if the fields in the expression calculation are coming from a joined layer, there are potential issues with how the expression is parsed. For example, if the join fields are coming from a csv, then the example above worked fine. However, if the join fields were coming from a shapefile, I had to change the line containing the expression from:

e = QgsExpression('Field_name_1 + Field_name_2')

to:

e = QgsExpression('"Field_name_1" + "Field_name_2"')
Ben W
  • 21,426
  • 3
  • 15
  • 39
  • I'm testing the code now - for the minute it creates the field ok but doesn't update the values - perhaps due to the joined fields - I'll keep trying and let you know – lhowarth Jan 24 '19 at 13:40
  • Bummer...does it work on the "native" fields? I tested it summing two integer fields & it was working fine. – Ben W Jan 24 '19 at 13:58
  • Ok, well same problem - works fine on 'native' fields but not on joined fields... no idea why - the script obviously works differently than the field calculater in the attribute table - would be good to know why! – lhowarth Jan 24 '19 at 14:37
  • 1
    That's disappointing. Not sure what's going on there. May be worth asking a new question. – Ben W Jan 24 '19 at 14:49
  • Yeah, I think i'll try and post that question later tomorrow - not being able to calculate from joined fields is a definately à pain... – lhowarth Jan 24 '19 at 18:50
  • 1
    @lhowarth, I did some testing and experienced a similar issue to you if the join layer was another shapefile (join fields from a csv worked fine). Looks like just an issue with how expressions are parsed. See my edit to my answer above. – Ben W Jan 26 '19 at 04:42
  • That is useful to know! Well spotted! – lhowarth Jan 28 '19 at 07:22
  • QgsVectorLayer.dataProvider().changeAttributeValues shouldn't be used within an edit session of the QgsVectorlayer (startEditing -> commitChanges) since it bypasses the edit block by accessing directly the underlying data source – Kalak Dec 07 '22 at 14:42
  • @Louis Cottereau, yep you're right- I changed it. This was quite an old answer for me. I have learnt a lot trying to answer questions on here over the years and no doubt made a few mistakes along the way. – Ben W Dec 07 '22 at 22:06
  • 1
    @BenW Same here, that is why when I see something that is not quite right on an answer I comment even if it might be old. It still might help someone else – Kalak Dec 08 '22 at 07:10
  • @Louis Cottereau, sure. Thanks for bringing it to my attention. Cheers. – Ben W Dec 09 '22 at 10:19
1

You can try:

layer = iface.activeLayer()
fieldlist = ['Field1','Field2','Sum1_2']
field_indexes = [layer.dataProvider().fieldNameIndex(f) for f in fieldlist]

layer.startEditing()
for feature in layer.getFeatures():
    _=layer.changeAttributeValue(feature.id(),field_indexes[-1], sum(feature.attributes()[field_indexes[0]:field_indexes[1]+1]))
layer.commitChanges()
BERA
  • 72,339
  • 13
  • 72
  • 161
  • Thanks, I thought fieldNameIndex was replaced in QGIS 3 ? That said, it I have tried your script and it does seem to work :) - partly - I nolonger have NULL values in the field but everything equals 0 - which is wrong - so I'm guessing it's to do with the field_indexes ? – lhowarth Jan 24 '19 at 10:59
  • @lhowarth Not sure why you get 0. I tried it on two different shapefiles with success. What is the field type of your fields? – BERA Jan 24 '19 at 11:23
  • @ BERA Field 1 and Field 2 are both integer64 my Sum1_2 field is straight Integer – lhowarth Jan 24 '19 at 11:45
  • 1
    @ BERA - they're all integers so i don't think it'll be that, but I'll try and make my Sum1_2 field an Integer64 and see if that changes the result – lhowarth Jan 24 '19 at 11:49
  • it doesn't seem to change the result – lhowarth Jan 24 '19 at 12:16
  • I just tried it on all int fields and it worked. Have you activated the layer by clicking it in table of contents? Are you only using my code or have you mixed it with your own? – BERA Jan 24 '19 at 12:22
  • Hi, I'm just using your script (but with field names modified) and yes layers was activated. 'layer = iface.activeLayer() fieldlist = ['field 1','field 2','Test'] field_indexes = [layer.dataProvider().fieldNameIndex(f) for f in fieldlist]

    layer.startEditing() for feature in layer.getFeatures(): _=layer.changeAttributeValue(feature.id(),field_indexes[-1], sum(feature.attributes()[field_indexes[0]:field_indexes[1]+1])) layer.commitChanges()' After the script runs it replaces the null values in the new field by 0 - so it finds the table and the new field...

     – lhowarth Jan 24 '19 at 12:31
  • hmmm, just wondering my Field 1 and Field 2 are actually produced by a join... i'll see if i have same problem if I use other fields which are "native" to the table – lhowarth Jan 24 '19 at 12:33
  • 1
    ok, so your script works if the fields are native to the table (i.e. not produced through a join).... – lhowarth Jan 24 '19 at 12:41
  • This is odd, if I create a new field and use the field calculater in the attribute table i don't have this problem with joined fields... – lhowarth Jan 24 '19 at 12:46
  • @lhowarth, after I posted my answer above, I read your comments about your problems updating your new column with a value calculated from joined fields. I will be interested to see if my code above works or whether you have the same problem. – Ben W Jan 24 '19 at 13:18