Youth baseball has been played in San Rafael since the 1880's. In 1937 Jacob Albert donated the land that became Albert Park for a sports field for young people. Crowds thronged to Albert Field on July 16, 1940 for a grand celebration of his civic accomplishments. Ray Brusati's accordion band played in his honor.

In the 1940's Park Superintendent Charlie Pinza along with San Rafael merchants organized games for the kids and helped create the San Rafael Athletic Association. This group organized leagues and teams for the kids. Some of the founding members included Rome Brusati, Vic Jones, Collin Hill and Angelo Columbo.

In 1954 the Association turned the baseball program over to the parents and they, in turn, joined Little League International. Games during this time were played only during summer vacation when school was not in session. Some of the early team sponsors were Taveggia, Brusati Sport and Saddle Shop (T&B Sports), Cerutti Bros. Service Station, San Rafael Fire Department and the San Rafael Rotary Club.

Games were later played at a field at 4th and Irwin. The site was built upon so the field was moved to 2nd and Lincoln. In the 1960's the field was moved back to Albert Park. At one point in time SRLL games were played on the grounds of San Quentin Prison.

On the Opening Day of the 2014 season, the 60th anniversary of the San Rafael Little League, a proclamation was issued by the mayor of San Rafael declaring March 15, 2014 as San Rafael Little League Day!