Need Advice on Handling High-Dimensional Data in Data Science Project

tress — Sun, 28 Apr 2024 11:57:46 GMT

Hey everyone,

I’m relatively new to data science and currently working on a project that involves a dataset with over 60 columns. Many of these columns are categorical, with more than 100 unique values each.

My issue arises when I try to apply one-hot encoding to these categorical columns. It seems like I’m running into the curse of dimensionality problem, and I’m not quite sure how to proceed from here.

I’d really appreciate some advice or guidance on how to effectively handle high-dimensional data in this context. Are there alternative encoding techniques I should consider? Or perhaps there are preprocessing steps I’m overlooking?

Any insights or tips would be immensely helpful.

Thanks in advance!

tress Tracker

Need Advice on Handling High-Dimensional Data in Data Science Project