intake_sklearn
Intake plugin to load Scikit-Learn model pickle files
- Declare locations of model pickle files
- Declare required packages and check for version compatibility
- Standardize model loading for various model types
- Scikit-Learn, and eventually XGBoost, Keras/Tensorflow, Dask-ML, etc.
Sklearn driver
import intake
model_source = intake.open_sklearn('optimized_model.pkl')
model_source<intake_sklearn.source.SklearnModelSource at 0x1143d7a50>
model = model_source.read()
model Pipeline(memory=None,
steps=[('standardscaler',
StandardScaler(copy=True, with_mean=True, with_std=True)),
('gradientboostingregressor',
GradientBoostingRegressor(alpha=0.9, criterion='friedman_mse',
init=None, learning_rate=0.1,
loss='ls', max_depth=3,
max_features=None,
max_leaf_nodes=None,
min_impurity_decrease=0.0,
min_impurity_split=None,
min_samples_leaf=1,
min_samples_split=2,
min_weight_fraction_leaf=0.0,
n_estimators=100,
n_iter_no_change=None,
presort='auto', random_state=None,
subsample=1.0, tol=0.0001,
validation_fraction=0.1, verbose=0,
warm_start=False))],
verbose=False)
Incompatible versions?
old_model_source = intake.open_sklearn('old_model.pkl')
old_model_source.read() ---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-3-5b1fd5628c1f> in <module>
1 old_model_source = intake.open_sklearn('old_model.pkl')
----> 2 old_model_source.read()
~/Development/intake_sklearn/intake_sklearn/source.py in read(self)
60 'but version {} has been installed in your current environment.'
61 ).format(self.metadata['sklearn_version'], sklearn.__version__)
---> 62 raise RuntimeError(msg)
63
64
RuntimeError: The model was created with Scikit-Learn version 0.20.1 but version 0.21.3 has been installed in your current environment.
Catalogs
Store models wherever FSSpec can access them. You will need also install appropriate packages (s3fs, etc.)
s3_model = cat.s3_model.read()data = [
85.5, # Max temperature
6, # Month
False, # Holiday
True, # Weekend
True # home game
]
value = s3_model.predict([data])array([448.81569766])
What's next?
-
Is pickle the best (or only format)?
- I've had luck with JSONPickle model files, except DecisionTree
-
Add more drivers
- XGBoost, Keras/Tensorflow, Dask-ML
-
ONNX getting better
-
pipeline viz PR in Sklearn
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent