大数据集清理：如何根据多个类别填写缺失数据并按行顺序搜索

问题描述 投票：4回答：1

这是我的第一篇StackOverflow帖子，所以我希望它不难理解。

我有一个大型数据集（~14,000）行观鸟。这些数据是通过站在一个地方（点）并计算你在3分钟内看到的鸟类来收集的。在每个点数内，新的鸟类观察成为新行，因此有许多重复的日期，时间，地点和点（站点内的特定位置）。同样，每个点数都是3分钟。因此，如果您在第1分钟看到黄色鸣鸟（编码为YEWA），则该特定点数（日期，地点，点和时间）将与MINUTE = 1相关联。 ID =观察者的intials和Number =被发现的鸟的数量（这里不一定重要）。

但是，如果看不到BIRDS，那么“NOBI”将进入该特定分钟的数据集。因此，如果整个3分钟点数有NOBI，它们将是具有相同日期，地点，点和时间的三行，并且对于三行中的每一行，在“BIRD”列中为“NOBI”。

所以我有两个主要问题。首先是一些观察者在三分钟内进入“NOBI”一次，而不是三次（每分钟一次）。在“MINUTE”空白（变为NA）和“BIRD”=“NOBI”的任何地方，我需要添加三行数据，除“MINUTE”之外的所有列都具有相同的值，该值应为1，对于各行，2和3。

如果它看起来像这样：

   ID     DATE SITE POINT TIME MINUTE BIRD NUMBER
1  BS 5/9/2018  CW2  U125 7:51     NA NOBI     NA
2  BS 5/9/2018  CW1  D250 8:12      1 YEWA     2
3  BS 5/9/2018  CW1  D250 8:12      2 NOBI     NA
4  BS 5/9/2018  CW1  D250 8:12      3 LABU     1

它看起来应该是这样的：

   ID     DATE SITE POINT TIME MINUTE BIRD NUMBER
1  BS 5/9/2018  CW2  U125 7:51      1 NOBI     NA
2  BS 5/9/2018  CW2  U125 7:51      2 NOBI     NA
3  BS 5/9/2018  CW2  U125 7:51      3 NOBI     NA
4  BS 5/9/2018  CW1  D250 8:12      1 YEWA     2
5  BS 5/9/2018  CW1  D250 8:12      2 NOBI     NA
6  BS 5/9/2018  CW1  D250 8:12      3 LABU     1

注意：如果您想将某些数据输入R控制台，我在本文末尾使用dput包含了一些内容，这应该比上面复制和粘贴更容易输入

我尝试复制if语句有多种条件（基于：R multiple conditions in if statement和Ifelse in R with multiple categorical conditions）我尝试写了很多方法，包括dplyr的管道，但是请参阅下面的一些代码，注释和错误消息的例子。

>if(PC$BIRD == "NOBI" & PC$MINUTE==NA){PC$Fix<-TRUE}
 Error in if (PC$BIRD == "NOBI" & PC$MINUTE == NA) { : 
   missing value where TRUE/FALSE needed
 In addition: Warning message:
 In if (PC$BIRD == "NOBI" & PC$MINUTE == NA) { :
   the condition has length > 1 and only the first element will be used

## Then I need to do something like this:
>if(PC$Fix<-TRUE){duplicate(row where Fix==TRUE, times=2)} #I know this isn't 
    ### even close, but I want the row to be replicated two more times so 
    ### that there are 3 total rows witht he same values
    ### Fix indicates that a fix is needed in this example
# Then somehow I need to assign a 1 to PC$MINUTE for the first row (original row), 
# a 2 to the next row (with other values from other columns being the same), and a 3 
# to the last duplicated row (still other values from other columns being the same)

对我来说似乎更难的第二个问题是按顺序搜索数据集，或者可能以某种方式搜索DATE，SITE，POINT和TIME。对于下一组日期，时间，站点和点，分钟值应始终从1 ...到2 ...到3，然后再返回到1。也就是说，每个点数应该具有1：3的所有值。但是，一个计数可能在MINUTE = 1中有多次目击，因此在MINUTE = 2之前有5或6（或20）MINUTE = 1。但是，再次，这个数据集中的一些观察者只是在没有BIRDS（NOBI）的情况下留下一行，而不是为每个MINUTE写一行BIRD =“NOBI”。那就是数据集：

   ID     DATE SITE POINT TIME MINUTE BIRD NUMBER
...
4  BS 5/9/2018  CW2  U125 7:54      1 AMRO      1
5  BS 5/9/2018  CW2  U125 7:54      1 SPTO      1
6  BS 5/9/2018  CW2  U125 7:57      1 AMRO      1
7  BS 5/9/2018  CW2  U125 7:57      1 SPTO      1
8  BS 5/9/2018  CW2  U125 7:57      1 AMCR      3
9  BS 5/9/2018  CW2  U125 7:57      2 SPTO      1
10 BS 5/9/2018  CW2  U125 7:57      2 HOWR      1
11 BS 5/9/2018  CW2  U125 7:57      3 UNBI      1

我们可以看到7:57点计数时间已经完成（MINUTE值为1：3）。但是，7：54点计数时间在MINUTE = 1时停止。意思是，我需要在下面再输入两行，它们具有所有相同的DATE，SITE，POINT，TIME信息，但是对于第一个添加的行，MINUTE = 2和BIRD =“NOBI”，MINUTE = 3，BIRD =“NOBI” “为第二个增加的行。所以看起来应该是这样的：

   ID     DATE SITE POINT TIME MINUTE BIRD NUMBER
...
4  BS 5/9/2018  CW2  U125 7:54      1 AMRO      1
5  BS 5/9/2018  CW2  U125 7:54      1 SPTO      1
6  BS 5/9/2018  CW2  U125 7:54      2 NOBI      NA
7  BS 5/9/2018  CW2  U125 7:54      3 NOBI      NA
8  BS 5/9/2018  CW2  U125 7:57      1 AMRO      1
9  BS 5/9/2018  CW2  U125 7:57      1 SPTO      1
10 BS 5/9/2018  CW2  U125 7:57      1 AMCR      3
11 BS 5/9/2018  CW2  U125 7:57      2 SPTO      1
12 BS 5/9/2018  CW2  U125 7:57      2 HOWR      1
13 BS 5/9/2018  CW2  U125 7:57      3 UNBI      1

最后，我理解这是一个漫长而复杂的问题，我可能没有很清楚地表达出来。如果需要澄清，请告诉我，我很乐意听到任何建议，即使它没有完全解决我的问题。先感谢您！

如果您想将数据样本输入R，则此行下方的所有内容仅对您有用

要将我的数据输入R控制台，请将所有内容从“结构”功能复制并粘贴到代码结尾，然后在R控制台中将其作为数据框输入，代码为：dataframe<-structure(list...请参阅Example of using dput()以获取帮助。

PC<-read.csv("PC.csv") ### ORIGINAL FILE
dput(PC)
structure(list(ID = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "BS", class = "factor"), 
DATE = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "5/9/2018", class = "factor"), 
SITE = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "CW2", class = "factor"), 
POINT = structure(c(2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = c("M", "U125"), class = "factor"), 
TIME = structure(c(8L, 8L, 8L, 9L, 9L, 10L, 10L, 10L, 10L, 
10L, 10L, 11L, 1L, 1L, 1L, 2L, 2L, 2L, 3L, 3L, 4L, 4L, 4L, 
4L, 5L, 5L, 5L, 5L, 5L, 5L, 6L, 7L), .Label = c("6:48", "6:51", 
"6:54", "6:57", "7:12", "7:15", "7:18", "7:51", "7:54", "7:57", 
"8:00"), class = "factor"), MINUTE = c(1L, 2L, 3L, 1L, 1L, 
1L, 1L, 1L, 2L, 2L, 3L, 1L, 1L, 2L, 3L, 1L, 2L, 3L, 1L, 1L, 
1L, 1L, 2L, 3L, 1L, 1L, 1L, 2L, 3L, 3L, NA, NA), BIRD = structure(c(6L, 
6L, 6L, 2L, 7L, 2L, 7L, 1L, 7L, 5L, 8L, 8L, 6L, 6L, 6L, 6L, 
6L, 6L, 7L, 7L, 7L, 7L, 6L, 8L, 3L, 7L, 9L, 5L, 4L, 2L, 6L, 
6L), .Label = c("AMCR", "AMRO", "BRSP", "DUFL", "HOWR", "NOBI", 
"SPTO", "UNBI", "VESP"), class = "factor"), NUMBER = c(NA, 
NA, NA, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, NA, NA, NA, NA, 
NA, NA, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, NA, 
NA)), class = "data.frame", row.names = c(NA, -32L))


PCc<-read.csv("PC_Corrected.csv")  #### WHAT I NEED MY DATABASE TO LOOK LIKE
dput(PCc)
structure(list(ID = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L), .Label = "BS", class = "factor"), DATE = structure(c(1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "5/9/2018", class = "factor"), 
SITE = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L), .Label = "CW2", class = "factor"), POINT = structure(c(2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = c("M", 
"U125"), class = "factor"), TIME = structure(c(8L, 8L, 8L, 
9L, 9L, 9L, 9L, 10L, 10L, 10L, 10L, 10L, 10L, 11L, 11L, 11L, 
1L, 1L, 1L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 4L, 4L, 4L, 4L, 5L, 
5L, 5L, 5L, 5L, 5L, 6L, 6L, 6L, 7L, 7L, 7L), .Label = c("6:48", 
"6:51", "6:54", "6:57", "7:12", "7:15", "7:18", "7:51", "7:54", 
"7:57", "8:00"), class = "factor"), MINUTE = c(1L, 2L, 3L, 
1L, 1L, 2L, 3L, 1L, 1L, 1L, 2L, 2L, 3L, 1L, 2L, 3L, 1L, 2L, 
3L, 1L, 2L, 3L, 1L, 1L, 2L, 3L, 1L, 1L, 2L, 3L, 1L, 1L, 1L, 
2L, 3L, 3L, 1L, 2L, 3L, 1L, 2L, 3L), BIRD = structure(c(6L, 
6L, 6L, 2L, 7L, 6L, 6L, 2L, 7L, 1L, 7L, 5L, 8L, 8L, 6L, 6L, 
6L, 6L, 6L, 6L, 6L, 6L, 7L, 7L, 6L, 6L, 7L, 7L, 6L, 8L, 3L, 
7L, 9L, 5L, 4L, 2L, 6L, 6L, 6L, 6L, 6L, 6L), .Label = c("AMCR", 
"AMRO", "BRSP", "DUFL", "HOWR", "NOBI", "SPTO", "UNBI", "VESP"
), class = "factor"), NUMBER = c(NA, NA, NA, 1L, 1L, NA, 
NA, 1L, 1L, 1L, 1L, 1L, 1L, 1L, NA, NA, NA, NA, NA, NA, NA, 
NA, 1L, 1L, NA, NA, 1L, 1L, NA, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
NA, NA, NA, NA, NA, NA)), class = "data.frame", row.names = c(NA, 
-42L))
r if-statement data-manipulation data-cleaning
1个回答
3
投票

这是使用dplyr元包中的tidyr和tidyverse来实现它的方法。

# Step one - identify missing rows.
#    For each DATE, SITE, POINT, TIME, count how many of each minute 
library(tidyverse)

# Convert factors to character to make later joining simpler, 
#   and fix missing ID's by assuming prior line should be used,
#   and make NOBI rows have a count of NA
PC_2_clean <- PC %>%
  mutate_if(is.factor, as.character) %>%
  fill(ID, .direction = "up") %>%
  mutate(NUMBER = if_else(BIRD == "NOBI", NA_integer_, NUMBER))


# Create a wide table with spots for each minute. Missing will
#   show up as NA's
# All the NA's here in the 1, 2, and 3 columns represent 
#   missing minutes that we should add.
PC_3_NA_find <- PC_2_clean %>%
  count(ID, DATE, SITE, POINT, TIME, MINUTE) %>%
  spread(MINUTE, n)

PC_3_NA_find
# A tibble: 11 x 9
# ID    DATE     SITE  POINT TIME    `1`   `2`   `3` `<NA>`
# <chr> <chr>    <chr> <chr> <chr> <int> <int> <int>  <int>
#   1 BS    5/9/2018 CW2   M     7:12      3     1     2     NA
# 2 BS    5/9/2018 CW2   M     7:15     NA    NA    NA      1
# 3 BS    5/9/2018 CW2   M     7:18     NA    NA    NA      1
# 4 BS    5/9/2018 CW2   U125  6:48      1     1     1     NA
# 5 BS    5/9/2018 CW2   U125  6:51      1     1     1     NA
# 6 BS    5/9/2018 CW2   U125  6:54      2    NA    NA     NA
# 7 BS    5/9/2018 CW2   U125  6:57      2     1     1     NA
# 8 BS    5/9/2018 CW2   U125  7:51      1     1     1     NA
# 9 BS    5/9/2018 CW2   U125  7:54      2    NA    NA     NA
# 10 BS    5/9/2018 CW2   U125  7:57      3     2     1     NA
# 11 BS    5/9/2018 CW2   U125  8:00      1    NA    NA     NA


# Take the NA minute entries and make the desired line for each
PC_4_rows_to_add <- PC_3_NA_find %>%
  gather(MINUTE, count, `1`:`3`) %>%
  filter(is.na(count)) %>%
  select(-count, -`<NA>`) %>%

  mutate(MINUTE = as.integer(MINUTE),
         BIRD = "NOBI",
         NUMBER = NA_integer_)


# Add these lines to the original,  remove the NA minute rows 
#   (these have been replaced with minute rows), and sort
PC_5_with_NOBIs <- PC_2_clean %>%
  bind_rows(PC_4_rows_to_add) %>%
  filter(MINUTE != "NA") %>%
  arrange(ID, DATE, SITE, POINT, TIME, MINUTE, BIRD)


# Check result
PC_5_with_NOBIs  %>%
  count(ID, DATE, SITE, POINT, TIME, MINUTE) %>%
  spread(MINUTE, n)

PC_5_with_NOBIs



# Now to confirm it matches your desired output.
#   Note, I convert to character to avoid mismatches between factors
PCc_char <- PCc %>%
  mutate_if(is.factor, as.character) %>%
  arrange(ID, DATE, SITE, POINT, TIME, MINUTE, BIRD)

identical(PC_5_with_NOBIs, PCc_char)
# [1] TRUE
最新问题
© www.soinside.com 2019 - 2024. All rights reserved.