Before you can work with data you have to get some. This course will cover the basic ways that data can be obtained. The course will cover obtaining data from the web, from APIs, from databases and from colleagues in various formats. It will also cover the basics of data cleaning and how to make data “tidy”. Tidy data dramatically speed downstream data analysis tasks. The course will also cover the components of a complete data set including raw data, processing instructions, codebooks, and processed data. The course will cover the basics needed for collecting, cleaning, and sharing data.
Working with Dates
Loading...
來自 Johns Hopkins University 的課程
Getting and Cleaning Data
5337 個評分
從本節課中
Week 4
Welcome to Week 4 of Getting and Cleaning Data! This week we finish up with lectures on text and date manipulation in R. In this final week we will also focus on peer grading of Course Projects.
與講師見面
Jeff Leek, PhDAssociate Professor, Biostatistics
Bloomberg School of Public Health
Roger D. Peng, PhDAssociate Professor, Biostatistics
Bloomberg School of Public Health
Brian Caffo, PhDProfessor, Biostatistics
Bloomberg School of Public Health
This lecture is about working with dates.
Dates can be sometimes a little bit tricky to work with as data
because they have all sorts of quirks like starting over after the end of a week or
starting over after the end of the month, and so forth.
So we're gonna start really simple, we're gonna look at the date function, so
this is the date function right here.
And if you call it, you'll actually get the time that is happening right then.
So this is the time that the slides were processed.
The class of this variable is actually just a character, so
the date function just returns a character that gives you the date and time.
But in general, it's not always so simple.
So for example, if you use the Sys.Date function,
you actually end up with something here that looks a little bit just like a date,
just like you saw before but maybe without the time.
And if you look at class of this variable here it's actually a Date variable.
So it has some different properties that make it a little bit nicer in terms of
analyzing Date data, but a little bit more difficult or
tricky than dealing with just text files in some respects.
So the first thing that we might wanna do is reformat that into a different format.
So, for example, we would use the format function.
We would apply it to that date.
And then we would say, we would like to see the abbreviation for
the day, the abbreviation for the month and then the actual date that you've seen.
So that gives you Sunday, January 12th just like this.
You can use different characters within these quotes, and so for
example you can get the day as a number.
You can get %a is abbreviated weekday, capital %A is unabbreviated weekday,
lowercase %m is month, 0 to 12 and so forth.
So you can use all of this to get the dates in ad format that is nice and
visually appealing to you.
You can also take character vectors like this.
So here is a character vector where I have a bunch of different dates.
And I can turn them into dates by telling the function
as that date that you should be looking for a day,
you should be looking for a month, and then you should be looking for a year.
And so what will then do is turn these into dates.
The nice thing about these is that then you can sort of manipulate them and
figure out how far apart they are among other things.
So for example, if I look at the difference in time between the first day
and the second day, there's a time difference of -1 days.
So if I subtract off the second day from the first day I'm one day behind.
And so you can actually turn that into a numeric variable, so
you can actually get the differences between dates in terms of the number of
days apart that they are very easily using dates.
You can also convert them to Julian as well as to other different variables.
So for example if you want to know what weekday it is, you can use the weekdays
function and it'll tell you that this date happened on a Sunday.
You can find the months so
that happened during January, and you can turn it into a Julian date.
And so what that is is the number of days that have occurred since the origin,
and it will tell you here what the origin is.
In this case, the origin is the first of January, 1970.
So that's the number of days that have occurred since the origin date.
The Lubridate package is actually very nice and makes
working with dates even simpler than it is with the ad.Date function, among others.
So if you load the Lubridate library then what you can do is convert a number
to a date and you can do that regardless of what the format is.
So if you tell it ymd, that means it's gonna look for
the year first followed by the month followed by the date in any
sort of the standard formats that you might imagine.
So in this case it's gonna find 2014, then 01 then 08 and turn that into a date.
You can also tell it to look for the month first, and then the day and
the year with the mdy function.
So that's month, day, year, and it will turn it into a date as well.
Or you can tell it to look for day, month, year, so
that's going to find the day as 03 and the month as 04 and the year as 2013.
So it's very nice because you don't have to actually format the dates or
the characters before you turn them into dates.
You can also do the same thing with time.
So you're going to start with year, month, day.
And then you can have an underscore and then hours, minutes,
seconds will turn this into a time, including hours, and minutes and seconds.
If this was in a different order, so if it started with seconds, then minutes,
then hours, you could make this ymd_, seconds, hours, minutes, smh.
And so you can also set the time zone.
So, here's a date, or a time, that you would like to turn into a date object,
and you can do that by setting the time zone to be a specific time zone.
And I'm getting this from this tutorial that was
on statistics.com about loop or date.
And so if you wanna learn more about how to set time zones and
what time zones are, you can look at ?Sys.timezone.
Some functions have slightly different syntax.
So if you're using the base date manipulation functions, you use weekday.
But if you are using the Lubridate package,
you're gonna use wday like this to get the weekday.
And this will give you sort of a number here which is gonna be three.
And then if you wanna know the weekday in terms of the actual abbreviation,
you're gonna have to say label=TRUE.
And then it'll tell you that it's Tuesday, the third day of the week,
according to the levels that have been defined for this variable.
So a lot more information is available on this very nice Lubridate tutorial from
which I borrowed some of the information for this.
You can also find similar content in the Lubridate Package Vignette.
And ultimately you want your dates to be of the class POSIXct or POSIXlt.
And so for more information type ?POSIXlt and
that'll give you a little bit more information about how to deal with dates.
This will become relevant later when we do some modeling
in the prediction class that deal with dates as well.
