Migrate from Pendo to PostHog

Last updated:

|Edit this page

Prior to starting a historical data migration, ensure you do the following:

  1. Create a project on our US or EU Cloud.
  2. Sign up to a paid product analytics plan on the billing page (historic imports are free but this unlocks the necessary features).
  3. Raise an in-app support request with the Data pipelines topic detailing where you are sending events from, how, the total volume, and the speed. For example, "we are migrating 30M events from a self-hosted instance to EU Cloud using the migration scripts at 10k events per minute."
  4. Wait for the OK from our team before starting the migration process to ensure that it completes successfully and is not rate limited.
  5. Set the historical_migration option to true when capturing events in the migration.

Migrating data from Pendo is a two step process:

  1. Export data via Pendo Data Sync
  2. Convert Pendo data to the PostHog schema and capture in PostHog

1. Export data via Pendo Data Sync

Pendo Data Sync enables you to export data to a warehouse like S3, Azure, or Google Cloud. This requires their highest Ultimate tier of pricing. See their docs for details on how to set it up.

This exports event, features, guides, Pages, and more in a .avro format which we can then convert and capture into PostHog.

Want to make this guide better (and a $75 merch code)? We're looking for sample Pendo data from their Data Sync or aggregations API to improve this guide. Email ian@posthog.com if you have access and are willing to share.

2. Convert Pendo data to the PostHog schema and capture in PostHog

The schema of Pendo's exported event data is similar to PostHog's schema, but it requires converting to work with the rest of PostHog's data. You can see details on Pendo's schema in their docs and events and properties PostHog autocaptures in our docs.

If you have done one single historical export, you can query the allevents.avro table to get the event data. With it, you can then go through each row and convert it to PostHog's schema. This requires converting:

  • Event names like load to $pageview.
  • Properties like url to $current_url
  • Event browserTimestamp to a ISO 8601 timestamp

Once this is done, you can capture the data into PostHog using the Python SDK or the capture API endpoint with historical_migration set to true.

Here's an example version of a Python script reading from an allevents.avro file:

Python
from posthog import Posthog
from datetime import datetime
import json
from avro.datafile import DataFileReader
from avro.io import DatumReader
posthog = Posthog(
'<ph_project_api_key>',
host='https://us.i.posthog.com',
debug=True,
historical_migration=True
)
key_mapping = {
'browser': '$browser',
'eventSource': '$lib',
'language': '$browser_language',
'latitude': '$geoip_latitude',
'longitude': '$geoip_longitude',
'elementPath': 'elements_chain',
'region': '$geoip_subdivision_1_name',
'url': '$current_url',
'userAgent': '$user_agent',
'country': '$geoip_country_name',
'remoteIp': '$ip',
'pollId': 'survey_id',
'pollResponse': 'survey_response',
'eventType': '$event_type'
}
event_mapping = {
'load': '$pageview',
'pollResponse': 'survey sent',
'click': '$autocapture',
'change': '$autocapture',
'focus': '$autocapture'
}
omitted_keys = [
"accountId",
"browserTimestamp",
"destinationStepId",
"eventId",
"version",
"matchableId",
"oldVisitorId",
"periodId",
"visitorId",
"accountId",
]
with open("pendo_events.avro", "rb") as file:
reader = DataFileReader(file, DatumReader())
for event in reader:
# Capture the event
properties = {}
for key, value in event.items():
if value is None:
continue
if key in omitted_keys:
continue
elif key in key_mapping:
properties[key_mapping[key]] = value
elif key == 'propertiesJson':
properties = json.loads(value)
for key, value in properties.items():
if key in key_mapping:
properties[key_mapping[key]] = value
else:
properties[key] = value
else:
properties[key] = value
# Convert milliseconds to seconds then to ISO 8601
timestamp = datetime.utcfromtimestamp(event['browserTimestamp'] / 1000)
event_name = event['eventType']
if event_name in event_mapping:
event_name = event_mapping[event_name]
distinct_id = event['visitorId']
posthog.capture(
distinct_id=distinct_id,
event=event_name,
properties=properties,
timestamp=timestamp
)
reader.close()

Questions?

Was this page useful?

Next article

Billing limits and alerts

To help you avoid surprise bills, PostHog enables you to set billing limits for each of our products. Setting a billing limit means we will stop ingesting and processing your data so you are not charged over the set limit. In other words, if you exceed the billing limit you set, your additional data is lost forever. To set a billing limit: Go to your organization's billing settings View the billing limit section at the bottom of the product and click "Set billing limit." Set your dollar limit…

Read next article