I have a degree in software engineering and share the same goal as yours.
Start with Informal Proofs
Truth be told, you can learn more about proof by reading introductory maths book at university level and start writing informal proofs. A good book recommendation will be Velleman's "How to Prove It: A Structured Approach". The maths discord server is a great place to ask about your maths problem, url: https://discord.gg/math.
If you're determine on continuing this path, I can share some of my experience after learning Coq for nearly two months, especially on how writing proofs in Coq is different from writing proofs on paper.
Coq vs Other Proof Assistants
I can't comment on the differences between Coq and other proof assistants because I'm not familiar with them. But Coq and Lean might be similar because they are based on the same underlying type theory: CoC. Isabelle/HOL is based on a simpler theory instead, but they excel in producing very readable proof documents using Isar.
Maths in Coq vs Traditional Maths
- Proofs written in Coq are formal; proofs on paper are informal.
In programming parlance, informal proofs are pseudocodes. They're high level and written only to communicate with other mathematicians. Informal proofs are good enough as long as the other party understands what you're trying to communicate. However, you're required to fill in a lot of details by yourself.
On the other hand, formal proofs in Coq are like writing algorithms in an actual programming language. It's similar to how you'll write programs in Standard ML, OCaml or Haskell. You have to be correct down to the finest details because, now, instead of convincing an intelligent person, you have to convince Coq that your proofs are indeed correct.
- Coq is based on constructive type theory, while the mainstream maths are based classical set theory.
It means that you need to be familiar with both constructive type theory and classical set theory if you're going to learn maths with Coq. Why? You need to understand classical set theory to understand mainstream maths; then constructive type theory to translate those knowledge into Coq.
There are two main differences between them: (i) constructive vs classical maths, (ii) type theory vs set theory.
Classical maths is what you learn from most maths textbook (e.g. including the Velleman book I recommend above). On the other hand, constructive maths is maths done without principle of excluded middle. No proof by contradiction as well. I don't know of other subtle differences.
In set theory, new mathematical objects are always defined as sets or elements of some sets. Type theory is a bit familiar to you if you've programmed in static typed functional programming language. New mathematical objects are introduced very much like how you define new data structures in programming languages.
- Mastery of Coq takes time!
Don't underestimate the time and efforts needed to master your tool! You can start right away with pen and paper with informal proofs. But since you're going to write maths in Coq, you need to learn how Coq works!
- You may need to keep your proofs updated!
This might be surprising. Your proofs written with pen and paper won't get outdated, but your proofs written in a proof assistant will! Why? Your proofs written in proof assistants are essentially programs, so it's tied to specific versions of your proof assistant.
It means that if your proof assistant introduces breaking changes, your proofs will no longer work with the newer version of the proof assistant. A notable example is the port of mathlib from Lean 3 to Lean 4.
So, maintainability and portability across versions is a factor of consideration during your proof development. It might be no different from software development at this stage.
Connect with People
Ask for help in Coq zulip or here. People are busy, but they are friendly and welcoming. Feel free to chat with me on Coq zulip, my username there is
Hope that my experience will be useful to you. Thanks!