Serverless Stack Update to IAM

[jayair]

Update July 19

The guide and the sample projects have been updated. A few links for reference:

• Tagged version before the change
• The merged PR
• Tagged version after the change

---

What is going on

We are updating the Serverless backend API to use IAM as an authorizer and React.js app to make signed API requests.

What is going to happen

Over the next few days we'll post the chageset for the proposed update to give people a chance to review it. And then we'll update Serverless-Stack.com and all the code for the demo project. We'll still leave the changeset up and tag the repo in case people still want to look at the older version of the tutorial.

What should I do

If you are currently working through the tutorial there will be a changeset to refer to. And use this comment thread for help if you have any questions.

Why the change

The current version of the tutorial uses the User Pool directly as an authorizer for the backend. While this works, the recommended way to do authorization in AWS is by using the IAM. All AWS API requests must be securely signed with Signature Version 4 using IAM credentials. Serverless Framework added support for IAM as an authorizer relatively recently. To use IAM as an authorizer we need to use our Cognito Identity Pool to grant our users temporary IAM credentials. This setup is also necessary in the case where you would like to add Google or Facebook as a way to authenticate your users.

Is this a big change

It is not a very big change to the code base of our demo project. But the flow is a little different and the two setups (the current one and the IAM one) cannot work together at the same time. Also, IAM as an authorizer is more secure but it can be a lot more confusing to set up and difficult to work through.

---

As always feel free to contact us via Twitter (@fanjiewang or @jayair) or email if you have any questions.

[dbeja]

That's great! Will you also include an example with a Social Identity Provider (Google, Facebook,...)?

[jayair]

@dbeja It's not in this update but we will work on that after we roll this out. We haven't been able to write about the social logins pretty much because we needed to do make this update first.

[mmiinnovations]

Thanks! This is great.

[jayair]

Proposed Chageset (WIP)

https://github.com/AnomalyInnovations/serverless-stack-com/compare/dev

The new changes are being worked on in the above brach. Feel free to take an early peek.

[larrybek]

Waiting for your update !!! You're doing great work.

[bharloe]

Love this tutorial. Any idea of a specific time frame we can expect these changes to get published?

[jayair]

@bharloe Yeah they are just about ready to be published. We just wanted to make sure we give folks who are going through the tutorial currently a bit of a heads up.

Ideally, it should be up early next week.

[bharloe]

Thanks, looking forward to it!

[zamirkhan]

Thanks for the heads up! For those who have already worked through most of the tutorial and made some customizations for their own specific needs, do these changes imply starting from scratch on the API side of things or is it something that can be migrated to with relative ease?

[jayair]

@zamirkhan Yeah you should be able to migrate. It's just a few changes to different parts of the setup.

[muescha]

i would like to start the tutorial - if it is ready to publish, why not publish it now and make the old tutorial available via subfolder / subdomain...

[jayair]

@muescha It's almost ready. Just need to wrap up the code samples for all the chapters. I'm aiming for Wednesday.

[CharithW]

This is great.
I have struggling through the net to find proper way to integrate with apigClient sdk to my react app.
Hopefully you will be able to release this soon.
Thank you for the great work :)

[jayair]

Serverless Stack has been updated to use IAM as an authorizer. A few links for reference:

• Tagged version before the change
• The merged PR
• Tagged version after the change

[huatzhi]

Will this guide include how to write tests for the system in the future? (e.g. with mocha or jasmine)

[larrybek]

@jayair wanted to ask you if you could help me with an issue I tried to upload S3 bucket nested folder with IAM example that you have provided but Im getting Network Forbidden. Do you have some ideas why it can be. I tried without nesting folder its working properly.

[jayair]

@larrybek The ARN we are using for S3 arn:aws:s3:::YOUR_S3_UPLOADS_BUCKET_NAME/${cognito-identity.amazonaws.com:sub}* is looking for files inside a given bucket pre-fixed the identity user sub (or user id for our case).

What does the path of your S3 bucket (with the nested folders) look like?

[jayair]

@huatzhi Yeah we are planning to. We do have a repo that we are using with tests running using Jest. If folks are interested then we can share that.

[larrybek]

@jayair I changed the identity pool policy to access nested folders and it worked. Thanks.

[rajeget]

@jayair the cognito is using customized authentication ? Also are you using Secure Remote Password (SRP) protocol. or planning to ?
Thanks for doing this 👍

[jayair]

@rajegit We are using the User Pools as the authentication provider for now. According to the AWS docs, the Cognito JS SDK (and iOS and Android) uses SRP for it's auth flow.

[zamirkhan]

@jayair I've tried to migrate the authentication changes, but am a bit stuck.

I know this is likely beyond the scope of what you're trying to provide, but I found it a bit difficult to navigate the markup diffs to try to find out what I needed to change, instead of having to go through the entire tutorial again (since it had been a while).

However, now that I thought I had made the required changes, although the "Test the API" step works for me.

I am getting 403 responses, both via the react app and my own Mocha.js tests for the API. I rigged the Mocha.js tests up to use the same authentication scheme (e.g. using your sigV4Client.js and aws helper). This is the response I get to my requests:

message: 'The request signature we calculated does not match the signature you provided.
Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been
(... omitted...)

The String-to-Sign should have been
(... omitted ...)

I've tried to check over all of my config files (id's, key's etc) and can't seem to find any obvious errors. The strange thing is it works using the manual testing suggested in the guide. Any ideas?

[jayair]

@zamirkhan This seems related to the way the request is being constructed. Can I see the part that makes the call to the helper to make the request and the helper function as well?

[zamirkhan]

@jayair Here is a gist of the code involved. If I missed anything you need, let me know. I'm using Chakram.js (which is based on Mocha.js and uses the request library) for testing.

https://gist.github.com/zamirkhan/e5e6284ceb522edf073b0439b5350a30

It seems that the "Test the API step" directly tests that lambda through API gateway, but not by making an HTTP request, is that right? I'd prefer to have these tests do the latter to test the full request pathway, in the same way that the React client will. If there's a better way to do that then what I am that would avoid these issues or make this simpler, let me know.

[jayair]

@zamirkhan It does make a normal HTTP request. It just makes it easier by calculating the security headers as a part of the process.

Just glancing at the gist, it looks okay but I guess the flow is different from the one we use in the React client (or in the Test APIs chapter). Any reason why you are using adminInitiateAuth?

[maletor]

This has got to be a regression, right?

Using AWS SDK to communicate with S3 directly? Sure, this makes enough sense, but ultimately, you probably want to do this through API Gateway for rate limiting features, etc.

Using AWS SDK to hit resources on API Gateway with Signature Version 4? I don't think so. Just use the Authorization header. What is your source for creating this huge internal library?

[jayair]

@maletor The AWS SDK is used to generate temporary IAM credentials. This is exactly as it was before.

Using the user token vs IAM as an authorizer depends entirely on your setup. If you are only going to talk to API Gateway then stick with the user token. But if you are going to use multiple AWS resources then IAM gives you better control over those resources. In this tutorial we want to cover the minimal setup but still allow people to build on it (something a lot of our readers do). Hence, the decision to cover the IAM setup.

[forrest-akin]

You may have just mentioned it (better control over access to AWS resources), but could elaborate (or provide a resource) on why IAM should be preferred over Cognito User Pools for auth in AWS?

[jayair]

@forrest-akin We'll be writing more on this soon but here is the gist of it. IAM auth is what AWS uses for all their resources. You see this in the tutorial with S3 where we need to get the IAM credentials. You also need the IAM auth if you are rolling out your own auth system using Federated Identities.

The special case here is if you are using only API Gateway and all your users are stored in the User Pool. For this case, AWS added the user token method of authentication (there are a few reasons why I think AWS has done this). If this applies to your setup, you don't need to worry about using IAM.

The part of IAM (in conjunction with the Cognito Identity Pool) that gives more control is evident in this chapter (https://serverless-stack.com/chapters/create-a-cognito-identity-pool.html) where we set the policy for a user based on their cognito-identity.amazonaws.com:sub. So we can decide what resources a user get's access to based on their Cognito Identity (regardless of if they are a User Pool, FB, Google, or custom auth user).

For these reasons we decided to show people how to use the IAM sigV4 way of authentication. In the most recent update (https://github.com/AnomalyInnovations/serverless-stack-com/releases/tag/v1.1) we refactored the code so that we authenticate using the User Pool and get the IAM credentials at the same time. This allows folks to use this setup and not be limited to the AWS resources they use or to the User Pool.

Finally, if you are only going to use API Gateway with the User Pool you can pass in the user token here (https://github.com/AnomalyInnovations/serverless-stack-demo-client/blob/v1.1/src/libs/awsLib.js#L36) instead of using sigV4. So the rest of your client code does not depend on the system you are using.

Hope that clears things up. There are a few other things here but we are going to be writing about that soon.

[sean9keenan]

Regarding this change, the size of the bundle was driving me a bit nuts - so I went and figured out how to minimize the bundled size of the aws-sdk. It should shave off over half of the size of the application bundle.

AnomalyInnovations/serverless-stack-demo-client#15