Welcome to the Treehouse Community

Want to collaborate on code errors? Have bugs you need feedback on? Looking for an extra set of eyes on your latest project? Get support with fellow developers, designers, and programmers of all backgrounds and skill levels here with the Treehouse Community! While you're at it, check out some resources Treehouse students have shared here.

Looking to learn something new?

Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and join thousands of Treehouse students and alumni in the community today.

Start your free trial

Data Analysis Cleaning and Preparing Data Handling Bad Data Duplicated Data

Posted by alexkalt
alexkalt
15,319 Points

Exercise from Stage 2 Video 5

  1. We import itertools
  2. Assign the returned boolean series denoting duplicate rows to the ind variable
  3. Get the unique SEQN values of the duplicated rows and assign them to the dup_id variable

After that I get lost...

  • Could someone explain what is happening step by step fro the beginning.
  • What does the np.where(demo['SEQN'] == id)[0] accomplish ?
  • What does the for (row1, row2) in itertools.product(dup_rows, repeat=2) accomplish ?
import itertools
ind = demo['SEQN'].duplicated(keep=False)
dup_id = demo.loc[ind,'SEQN'].unique()
for id in dup_id:
    dup_rows = np.where(demo['SEQN'] == id)[0]
    for (row1, row2) in itertools.product(dup_rows, repeat=2):
        demo.iloc[row1,:] = demo.iloc[row1,:].fillna(demo.iloc[row2,:], axis=0)