When to do feature encoding with XGBoost, random forest

[pippo]

Hi I would like to understand when it is necessary to do feature encoding. For example, I am using models like XGBoost and Random Forest. Do I need to encode categorical variables? For instance, if I have a variable like ‘education’ with levels: high-school, bachelor, master, phd, should I recode the feature as: high-school=1, bachelor=2, master=3, phd=4? Or is it better to use one-hot encoding, creating 4 new columns? What if the categorical variable doesn’t have a natural order, like ‘country of birth’? In this case, is it correct to only apply one-hot encoding?

[mlauber71]

@pippo This will very much depend on your task. XGBoost and random Forest can handle categorical data themself. Although advanced data preparation can help to improve performance.

Data Manipulation of categorical data in the background in ML process

@moritz_skb every KNIME nodes has some sort of documentation like the XGBoost Tree Ensemble Learner: [image] There you will find additional links and literature to explain what has been done. In this case to the official documentation and there you will find what the algorithm does: https://xgboost.readthedocs.io/en/stable/tutorials/categorical.html The KNIME implementation might ‘compress’ such settings or provide some of them by default or with switches in the node. If you want to do the…

More on Data Preparation and Machine Learning you can find here:

KNIME — Machine Learning and Artificial Intelligence— A Collection

There are more advanced preparation techniques like vtreat and other Python packages. Some examples are here (I have some code still on my machine that might get published in the future):

ashokharnal/Collection of Components and Workflows – feature-generation