Hey Brian, great question.
Yes, you are testing to see if you have autoscaling and concurrency settings appropriately set up, but you are also testing to make sure that your application handles doesn't have any bottlenecks.
What you'll find out quickly by running a load test is how your application handles throttling.
For us, we had an issue with the way we built a lookup, and IAM was getting throttled under load - causing our application to not function at all.
After a quick fix, everything started working as it should. But we learned a valuable lesson in how we use AWS services.