The speaker is James, our Savior's brother. "They" is the Jews, and "you" is the apostle Paul.
Verse 25 is a quotation taken from the conference in Acts 15, and the subject, according to verse 21, is the customs. The controversy did not involve the civil laws or the Ten Commandments. Instead, it involved the ceremonial additions, as is clearly shown in context by what Paul did.
The context shows what these customs were. Paul made the offerings required at the conclusion of a vow. It is clear that the passage is speaking about the ceremonies. It is also entirely possible that the controversy over these customs also involved the oral traditions of the Pharisees, which they were so devoted to.
There is no evidence that Paul ever taught any Jew to forsake Moses. To do so, he would have to preach against God. There is no evidence that Paul ever told them, "Do not circumcise your children." He certainly preached that keeping the law could not justify a person before God. His writings clearly state that we are justified by grace through faith in the sacrifice of Jesus Christ (Ephesians 2:8).
Plainly, Paul's own actions in Acts 21 testify that, though salvation or justification could not be won through keeping these things, keeping them was not destructive unless one depended upon them for justification or salvation. In addition, there was no hesitation on Paul's part to do them. Scripture gives no indication that he argued with James; in fact, we see a unity of mind between them. There is no indication of reluctance either, that somehow it would destroy Paul's faith in Jesus Christ, or that it would compromise him in the eyes of any Christian, Jew or Gentile, who might witness it.
This teaches that first-century Christians understood this issue. They clearly understood what we seem to have such a difficult time understanding nineteen centuries later. Nothing this God of love that we worship requires of us is bad for us. Sometimes what He requires may be difficult to bear, but it is not destructive to His purpose or thoughtless in any way. It is always intended to strengthen us.