Knime_io and DataFrame

[kamtaot]

Hi All,

I am new to Knime and Python as well. However, with the help of Knime teams, I am trying to query AD LDAP and I could able to reach this level. Now, finally, I got struck with the last statement. Any help is highly appreciated.

import pandas as pd
import knime_io as knio
from ldap3 import Server, Connection, ALL, NTLM, SUBTREE

bmuri = 'ldaps://XXXXXX:636'
bmusername = 'YYYYY'
bmpassword = 'ZZZZZZ'
bmsearch_base = 'DC=abc,DC=cde,DC=com'
bmsearch_filter = '(&(cn=*)(objectClass=user)(!(objectClass=computer)))'
bmattributes = ['cn', 'displayName', 'sAMAccountName', 'userAccountControl']
dictionary = []

try:
    server = Server(bmuri, get_info=ALL)
    conn = Connection(server, user=bmusername, password=bmpassword, authentication=NTLM, auto_bind=True)
except Exception as server_connection_error:
        print(str(server_connection_error))

entry_list = conn.extend.standard.paged_search(
    search_base = bmsearch_base, search_filter = bmsearch_filter, search_scope = SUBTREE, attributes = bmattributes, paged_size = 500, generator=False
)

for entry in entry_list:
    for key, value in entry.items():
        if key == 'attributes':
            ##print(value,"$%$%$%$%")
            dictionary.append(value)
            break

print(type(dictionary))
df = pd.DataFrame(dictionary)
print(type(df))
knio.output_tables[0] = knio.write_table(df)

From the above code, I am able to execute until the knio.output_tables[0] and it is displaying the results. For the type of dictionary, it is mentioning that it is list as below.
<class ‘list’>
For the df it is printing the statement as below.
<class ‘pandas.core.frame.DataFrame’>

However, the problem is there in the last statement. which is in output table. When I execute this statement, I get the below error.

Executing the Python script failed: Traceback (most recent call last):
File “”, line 34, in
File “D:\DAE\Software\Knime\plugins\org.knime.python3_4.6.2.v202209150850\src\main\python\knime_table.py”, line 507, in write_table
return _backend.write_table(data, sentinel)
File “D:\DAE\Software\Knime\plugins\org.knime.python3.arrow_4.6.1.v202207221015\src\main\python\knime_arrow_table.py”, line 377, in write_table
return ArrowWriteTable(self._create_sink(), data, sentinel)
File “D:\DAE\Software\Knime\plugins\org.knime.python3.arrow_4.6.1.v202207221015\src\main\python\knime_arrow_table.py”, line 326, in init
self._put_table(data, sentinel)
File “D:\DAE\Software\Knime\plugins\org.knime.python3.arrow_4.6.1.v202207221015\src\main\python\knime_arrow_table.py”, line 258, in _put_table
data = kap.pandas_df_to_arrow(data)
File “D:\DAE\Software\Knime\plugins\org.knime.python3.arrow_4.6.1.v202207221015\src\main\python\knime_arrow_pandas.py”, line 75, in pandas_df_to_arrow
return pa.Table.from_pandas(data_frame)
File “pyarrow\table.pxi”, line 3457, in pyarrow.lib.Table.from_pandas
File “D:\DAE\Software\Miniconda3\envs\py3_knime\lib\site-packages\pyarrow\pandas_compat.py”, line 607, in dataframe_to_arrays
arrays[i] = maybe_fut.result()
File “D:\DAE\Software\Miniconda3\envs\py3_knime\lib\concurrent\futures_base.py”, line 439, in result
return self.__get_result()
File “D:\DAE\Software\Miniconda3\envs\py3_knime\lib\concurrent\futures_base.py”, line 391, in __get_result
raise self.exception
File “D:\DAE\Software\Miniconda3\envs\py3_knime\lib\concurrent\futures\thread.py”, line 58, in run
result = self.fn(*self.args, **self.kwargs)
File “D:\DAE\Software\Miniconda3\envs\py3_knime\lib\site-packages\pyarrow\pandas_compat.py”, line 581, in convert_column
raise e
File “D:\DAE\Software\Miniconda3\envs\py3_knime\lib\site-packages\pyarrow\pandas_compat.py”, line 575, in convert_column
result = pa.array(col, type=type, from_pandas=True, safe=safe)
File “pyarrow\array.pxi”, line 312, in pyarrow.lib.array
File “pyarrow\array.pxi”, line 83, in pyarrow.lib._ndarray_to_array
File “pyarrow\error.pxi”, line 123, in pyarrow.lib.check_status
pyarrow.lib.ArrowTypeError: (“Expected bytes, got a ‘list’ object”, ‘Conversion failed for column displayName with type object’)

[mlauber71]

@kamtaot welcome to the KNIME forum. I think you cannot directly convert dictionaries or lists back to knime you will have to convert them to strings and untangle them later.

Here is an example with dictionaries

KNIME/Python - write and read back a dictionary – mlauber71

[kamtaot]

Thank you @mlauber71,

your suggestion has helped me think in different way and I converted the individual items to string and added to the dictionary. It worked after that. I changed the last few lines of the code as below.

for entry in entry_list:
    for key, value in entry.items():
        if key == 'attributes':
            l = dict()
            for bmkey, bmvalue in value.items():
                l[bmkey] = str(bmvalue)
            dictionary.append(l)
            break

knio.output_tables[0] = knio.write_table(pd.DataFrame(dictionary))