I took the in-person version of this course in Fall 2018, and it's definitely approachable for someone who doesn't yet know any linear algebra or remembers calculus all that well. You just start doing things with deep learning right away. After these quick wins, you'll hopefully get some more intuitive breakthroughs of what's actually happening. Then and only then do they suggest digging into the math more deeply (I'm still doing this last part).