Can I use a battery charging IC chip rated for between 4.5 and 6.5 volts to charge a 3.7 volt battery

Short Answer: No.

Long Answer: The safe thing is to always check the whitesheet for the battery against the whitesheet for the charging IC. Without knowing exactly what chemistry the battery is (guessing Li-Ion with a LiCoO2 or LiMnO2 cathode) and what the IC is for (guessing maybe three NiMH cells in series), potential alone isn’t a good indicator of charging parameters. Different chemistries like different current/potential profiles.