Problem in Append using pandas (Python)

I have added a python 1->1 node in KNIME, I would like to scan a paragraph from several text files (only 2 files can match) with some specific words at the beginning and end of paragraph. However, I cannot append the output correctly, i.e. the new row replace the original row. Here is the program. Kindly help:

import re
import os
import os.path
import pandas as pd
paralist = []
result = {}
OUTPUT_DIR = 'C:\Final Output\Testing_Output'
 
def test_patterns4(x, y, text, patterns=[]):
 global pyOut
 for pattern, desc in patterns:
  print 'Pattern %r (%s)\n' % (pattern, desc)
  #print ' %r' % text
  for match in re.finditer(pattern, text):
   s = match.start()
   e = match.end()
   substr = text[s+x:e-y]
   para=substr
   filepath = os.path.join(OUTPUT_DIR, file)
   output_file = open(filepath, 'w')
   output_file.write(para)
   result={'Para': para, 'file': file}
   pyOut = pd.DataFrame(result.values()).T
   pyOut = pyOut.append(pyOut,ignore_index=True)
  print
 return
 
x='First Word'
print(len(x))
y='Last Word'
print(len(y))
 
for root, dirs, files in os.walk('C:\Final Output\Testing\\'):
 for file in files:
  z = open(os.path.join(root, file),'r').read()
  test_patterns4(len(x), len(y), z,[ ('(?ims)First Word.*?Last Word', 'Testing')])
     
output_table = pyOut

The output from the python node generated two rows with same value, i.e. the first row was replaced by the second row. Since, I have exported the matched records to OUTPUT_DIR for checking, the outputs are exported correctly, but in two files. Thanks for help Lawson

Hey Lawson,

I think the problem lies in the following lines:
pyOut = pd.DataFrame(result.values()).T
pyOut = pyOut.append(pyOut,ignore_index=True)

You assign a new dataframe containing only your latest result to pyOut there and then append it to itself. If you change those lines for example to:

tmpOut = pd.DataFrame(result.values()).T
pyOut = pyOut.append(tmpOut,ignore_index=True)

everything should work as expected.

Best

Clemens