His service in New York regiments was during the Civil War and his service in 25th US Infantry Rgt was post-Civil War. The highest number regular army infantry regiment during the Civil War was 19th.
PROBABLY his service in 1st & 2nd US Cavalry Rgts. was also post-Civil War since they were also regular army units. His regular army service may explain why he was living in California.