Those items after your eg are part of the problem why story points don't work for you.
You really do need to control the amount of time your devs get distracted by other things. If they're solely focusing on that one project, you'll be fine.
Its why i personally appreciate XP more than plain old scrum.
This, 100%. When engineers have focus, no context switching... things get done. When they're interrupted with useless Zoom meetings every 2 hours, productivity drops off the cliff. If you have 2 meetings 1 hour apart, that 1 hour is basically useless: by the time you're in the zone, you're interrupted again.
You really do need to control the amount of time your devs get distracted by other things. If they're solely focusing on that one project, you'll be fine.
Its why i personally appreciate XP more than plain old scrum.