Boyfriend has two small children ages 3 and 4?

You need to try to get back your former attitude of welcoming the children. If you want to keep a loving relationship with your boyfriend, don't come between him and his other children. A man needs to take care of all of his children, and not only financially. You can be honest with him about your feelings, but what do you want or expect him to say, OK, the children won't come here if you don't want them to. Where do you want him to go to spend time with the kids? Too often the kids of the former relationship get the shaft (even if they get support money) while the kids the parent is living with currently come first - even the ones that are not his. I'm sure it won't be easy, but you knew about his kids and they need all the love and support they can get, not rejection.