python


Pandas: set a default datetime for None values


I have a Pandas dataframe with columns that contain dates as strings (in SQL-like format). However, many cells contain None values. I'm trying to convert these columns to Pandas dates using to_datetime and set a "default" value for cells that contain the None value. Example code below:
import pandas as pd
>>> d = {'a': [1,2,3],
'd1': ['2016-01-01','2015-10-02',None],
'd2': [None,'2016-04-03',None]}
>>> df = pd.DataFrame(d)
>>> print df
a d1 d2
0 1 2016-01-01 None
1 2 2015-10-02 2016-04-03
2 3 None None
>>> date_cols = ['d1','d2']
>>> df[date_cols] = df[date_cols].apply(pd.to_datetime)
>>> print df
a d1 d2
0 1 2016-01-01 NaT
1 2 2015-10-02 2016-04-03
2 3 NaT NaT
It's simple enough to convert the valid strings to a date, I just want to replace the NaT with the default_date. This is what I'd like the final dataframe to look like:
>>> default_date = '2015-01-01'
>>> print df
a d1 d2
0 1 2016-01-01 2015-01-01
1 2 2015-10-02 2016-04-03
2 3 2015-01-01 2015-01-01
use fillna
df[date_cols] = df[date_cols].fillna(pd.to_datetime('2015-01-01'))
df

Related Links

xlwt: 'CommonToken' object has no attribute 'txt'
Python repeat while user enters negative number
Assign different operators during python loop
When installing through pip, how do I tell matplotlib how to find tkinter?
Regex for weekdays in python [closed]
Detect Changes to Environment Variables
Select all by type: Geometry. Equivalent Python script?
Sorting algorithm doesn't work
How to avoid “Name 're' is not defined” error while connecting pycharm remote debugger?
Python - Generate binary list with itertools
Capture domain and path from URL with regex
Pylab animation multiple figures
Django Password Reset Email Error 10060
Excel delimited file
Why is scrapy dumping thousands of `ERROR` log messages without any description of the error?
Python 2.5.4: how to find sum of logarithm values

Categories

HOME
vue.js
sidekiq
jenkins-plugins
erlang
oauth
angular-cli
jsviews
blockchain
raspbian
decorator
reportportal
wms
visualization
couchdb-2.0
docker-windows
android-viewpager
getorgchart
x11
ssr
wordpress-theming
hashicorp-vault
glpk
ida
ada
mousewheel
jive
multiple-columns
raphael
cruisecontrol.net
jquery-form-validator
jquery-waypoints
classpath
multichoiceitems
ivy
ioc-container
body-parser
mapguide
nodeclipse
discord.net
linq-to-entities
cross-entropy
floating-accuracy
bayesian-networks
productivity
microdata
android-maps-v2
pytest-django
onmouseover
chown
crash-reports
volume
gce
butterknife
launch
pycurl
python-hypothesis
jcreator
spyne
push-diffusion
audioqueue
uiprogressview
amf
clob
cmocka
approval-tests
hateoas
geodjango
registrykey
adobe-indesign
web-component-tester
magento-1.12
start-job
java-collections-api
aquafold
boost-test
funkload
lumx
late-static-binding
chrome-for-android
baucis
angulartics
fogbugz-api
appfog
sametime
gdataxml
angularjs-timeout
ubuntu-11.10
web-safe-fonts
n-layer
responsetext
hobo
separation-of-concerns
serp
dashcode
email-spec
out-of-browser
msn
trampolines
filetime
method-signature
configurable

Resources

Mobile Apps Dev
Database Users
javascript
java
csharp
php
android
MS Developer
developer works
python
ios
c
html
jquery
RDBMS discuss
Cloud Virtualization
Database Dev&Adm
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App