Colorectal cancer (CRC) is the third most common cancer worldwide. Approximately 50% of these patients will develop liver metastases. Neoadjuvant chemotherapy is used to reduce size of metastases so that patients are eligible for potentially curative resection. Our study demonstrates that baseline CT scans can be used to predict response prior to initiation of chemotherapy. We analyzed baseline CT images of 342 patients with unresectable colorectal liver metastases (CRLM) who received chemotherapy at Memorial Sloan Kettering Cancer Center (MSK) or University of Texas MD Anderson Cancer Center (MDA). We predicted response as defined by Response Evaluation Criteria in Solid Tumors (RECIST). Different classifiers were evaluated, and the three best-performing ones were selected. Further, these were later combined in a stacking classifier to improve prediction accuracy. Naive Bayes showed the best performance with an accuracy of 0.745 and AUC of 0.785 when trained on features extracted from all tumors. The stacking classifier demonstrated a slightly better precision (0.742) and specificity (0.742). Logistic Regression was the best model for the largest tumor, as determined by 3D volumetric measurement, with an accuracy of 65%. These models outperformed neural networks in all trials.