Skip to content

Commit

Permalink
fix(harvest): incremental sync for harvest users (#105)
Browse files Browse the repository at this point in the history
## Describe your changes
Harvest has an `updated_since` param

## Issue ticket number and link

## Checklist before requesting a review (skip if just adding/editing
APIs & templates)
- [ ] I added tests, otherwise the reason is:
- [ ] External API requests have `retries`
- [ ] Pagination is used where appropriate
- [ ] The built in `nango.paginate` call is used instead of a `while
(true)` loop
- [ ] Third party requests are NOT parallelized (this can cause issues
with rate limits)
- [ ] If a sync requires metadata the `nango.yaml` has `auto_start:
false`
- [ ] If the sync is a `full` sync then `track_deletes: true` is set
- [ ] I followed the best practices and guidelines from the [Writing
Integration
Scripts](/NangoHQ/integration-templates/blob/main/WRITING_INTEGRATION_SCRIPTS.md)
doc
  • Loading branch information
khaliqgant authored Nov 11, 2024
1 parent 436500e commit 8f74f8f
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 7 deletions.
2 changes: 0 additions & 2 deletions flows.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3365,8 +3365,6 @@ integrations:
description: |
Fetches the list of users in Harvest
endpoint: GET /users
sync_type: full
track_deletes: true
runs: every day
output: User
scopes:
Expand Down
2 changes: 0 additions & 2 deletions integrations/harvest/nango.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,6 @@ integrations:
description: |
Fetches the list of users in Harvest
endpoint: GET /users
sync_type: full
track_deletes: true
runs: every day
output: User
scopes:
Expand Down
13 changes: 10 additions & 3 deletions integrations/harvest/syncs/users.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,6 @@ export default async function fetchData(nango: NangoSync): Promise<void> {
const config: ProxyConfiguration = {
// https://help.getharvest.com/api-v2/users-api/users/users/#list-all-users
endpoint: '/v2/users',
params: {
is_active: 'true'
},
paginate: {
type: 'link',
response_path: 'users',
Expand All @@ -30,6 +27,16 @@ export default async function fetchData(nango: NangoSync): Promise<void> {
retries: 10
};

const params: Record<string, string> = {
is_active: 'true'
};

if (nango.lastSyncDate) {
params['updated_since'] = nango.lastSyncDate.toISOString();
}

config.params = params;

for await (const harvestUsers of nango.paginate<HarvestUser>(config)) {
const users = harvestUsers.map(toUser);

Expand Down

0 comments on commit 8f74f8f

Please sign in to comment.