>What if my landlord tries to harass or threaten me?
There is a risk your landlord might try to do something stupid. You look a lot more sympathetic as a victim of harassment, and this can substantially increase your payout. All this said, don't be dumb - be armed in case they try anything and by California law, if they attempt to break in, you may use lethal force against them if you feel your life is in danger. However, do not reveal that you're armed or anything of the sort unless you truly need to in order to defend your life. Brandishing, etc. when you are not in actual danger is a good way to get the police interested, which you do not want. In any case, if you have a corporate landlord, it is unlikely they will do all that much but put a notice up on your door.
>Isn't this his property? What stops him from just coming in?
Real property is a bundle of sticks. When a landlord leases to you, or to someone who then assigns the lease to you, he hands you the stick of possession, or equitable title. You effectively own the property as against the whole world, including the person holding legal title. He is nothing but a trespasser on your property, and you can deal with him as if he is nothing but a trespasser if he attempts to come in and fuck with you. While he can, by statute, enter upon giving 24 hours notice, he can only do so if he has a good reason, and he cannot do so if doing so would lead to a breach of the peace even if he does have a good reason.
>How am I going to get a good reference?
Usually, the landlord will agree to a positive or neutral reference as part of any settlement. If all else fails, you can simply hop on to a roommate situation and get friends to vouch for you/get an AirBNB, if you can't get good formal references or fake them. Then, use those references to push yourself into your next situation.