Just to be precise, it's needles that are the worry. If you use syringes with removable needles, they don't present the same problem.
If you re-cap your needles, you barely need to give it any more thought.
The most realistic danger is to either yourself to trash disposal folks if a cap falls off while handling them. 99% of the time, if you cap your needles (or fixed-needle syringes) and store them so that their caps can't fall off somehow, there's no real danger of puncture. The caps on separate needles don't really come off accidentally, but some of the caps on insulin syringes aren't the best. Putting those in a 2L bottle with a screw top, or a box that you tape up when you're done, is about all you need to do as long as you re-cap them after use.
IV drug users have a different set of worries, because they sometimes draw blood back into the barrel... used to be called "booting and jacking." If you expose your syringe to your blood, you should consider it to be a biohazard, and take the same precautions as with the needles. Otherwise, you can toss 'em in the regular trash.