time-series-foundation-models / lag-llama Goto Github PK
View Code? Open in Web Editor NEWLag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting
License: Apache License 2.0
Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting
License: Apache License 2.0
Thanks for your great work. I'm looking forward to training it on my custom data.
When will you release the finetuning code?
Hi, thanks for the model and instructions. I could not get the idea of predicting the last part of the data. I tried zero-shot learning with my own data. Let's say I have 1 week of hourly data. When I want to use the "get_lag_llama_predictions" function, the "forecasts" object has the same timestamp as the last part of the given data. But I want to predict the future timestamps. I want to give 1 week of data to model and predict the next 2 days. How can I achieve that? Thanks.
I used best checkpoint after turning then used get_lag_llama_predictions
function to predict from best checkpoint . However, the predicted result is much lower than when I turned and trained that checkpoint.
First , thank you for the great job , still did not finish the paper but trying to see what's the embedding space of time series looking like .
It does not seem obvious how to get them out especially with the gluonTS wrapper , any thoughts here ?
Thank you !
Hi all,
Thank you for open-sourcing this library.
I am working on the task of anomaly detection in multivariate time series data. I would like to know how to use Lag-Llama to accomplish this. Is it possible for Lag-Llama to achieve this through fine-tuning?
Thanks!
Hello and thank you for the model. I'm experimenting with my dataset, and one kind of systematic error I found are charts like below, where the first prediction point is way off the latest known point from the context. I'm wondering what could be the reason for this and if the problem could be alleviated somehow? Many thanks.
Hi,
The current fine-tuning notebook on colab uses gluon-ts datasets.
Can you quickly demonstrate fine-tuning for any open-source dataset in a pandas dataframe?
That will be helpful for the larger audience.
Thanks.
Can the Taiwan Lottery accurately predict the next draw based on the results of the past ten draws with a 95% accuracy rate?
In my experiment, I need to compare the difference and accuracy of specialized forecasting models and general time series forecasting models.
Hi,
I would like to know the exact location in the lag-llama source code of the last hidden state to capture embeddings.
Regards,
Surendra
The loss calculation during training uses the probability density function loss of all targets in context and predict, rather than just predict. Why?
I am having bad accuracy (MSE of 2500+)
I am giving the model 10 days worth of data and using the default settings given in the updated colab with a wide dataframe.
Prediction length = 24
While i see improvement if i increase the prediction length i.e. inplace of predicting for 1 day i predict for 72 time stamps (3 days) that i can get a mse of 400. Which is still high and not as good.
This image is 3 days prediction:
Do let me know if i am making a large error when giving this data which is causing this low accuracy.
I have a dataset with only 1 time series array, so item_id
is always the same, which is equivalent to not to use it. On the Demo you mention:
The item_id is required only when your dataset has multiple series.
However its not working when not specifying this parameter, similarly when using PandasDataset(dfs_dict, target="target")
. Do you have any suggestions on how to adapt the data on this case?
Thanks a lot
Hello. Thank you so much for the open-source implementation.
I have a custom dataset with cryptocurrency values for daily prediction which except from the target values,
includes also 3 columns of related features.
Those columns could be included in your implementation? Are features taken into account from the model or is just the target value?
I noticed that the dimensions of the output of the first term in the list of:
forecast_it, ts_it = make_evaluation_predictions( dataset=backtest_dataset, predictor=predictor, )
is always (100, prediction_length), at least with the example and when using my own dataset, independent of the length of the test data length. Am I right here? Is this a bug? If not, how should I interpret the 100?
This the link to the Jupyter Notebook used for my own dataset https://drive.google.com/file/d/1WThIccnVKq_V48bC2FzmdzvHLEquLK94/view?usp=sharing
Kind Regards,
Hugo
Congrats on this foundation model.
Your publication doesn't mention anything about training time, hardware requirements, and costs. Could you elaborate on this?
Best regards
Thanks for such a great open source work. I feel so excited. I can’t wait to ask how long it will take to update the code for fine-tuning and training,You can close this issue at any time, it's just a little complaint from those who are waiting.
Hi,
Thank you for publishing the pretraining and finetuning scripts! They are really helpful.
For a university project, we are trying to reproduce the results from the paper. However, running the pretrain script, we observe very slow training speeds (~1 minute per epoch) on our hardware.
Running the pytorch profiler for 16 training batches, we see the following:
Action | Mean duration (s) | Num calls | Total time (s) | Percentage % |
---|---|---|---|---|
Total | - | 1397 | 99.734 | 100 % |
run_training_epoch | 91.657 | 1 | 91.657 | 91.901 |
[_TrainingEpochLoop].train_dataloader_next | 5.2931 | 16 | 84.689 | 84.915 |
[_EvaluationLoop].val_next | 0.246 | 19 | 4.674 | 4.6865 |
[LightningModule]LagLlamaLightningModule.optimizer_step | 0.10731 | 16 | 1.717 | 1.7216 |
run_training_batch | 0.10731 | 16 | 1.717 | 1.7216 |
[Strategy]SingleDeviceStrategy.training_step | 0.091875 | 16 | 1.47 | 1.4739 |
[Strategy]SingleDeviceStrategy.validation_step | 0.044368 | 19 | 0.843 | 0.84525 |
[Strategy]SingleDeviceStrategy.backward | 0.0135 | 16 | 0.216 | 0.21658 |
[Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_train_epoch_end | 0.141 | 1 | 0.141 | 0.14138 |
[Callback]ModelCheckpoint{'monitor': 'val_loss', 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_train_epoch_end | 0.093 | 1 | 0.093 | 0.093248 |
[LightningModule]LagLlamaLightningModule.transfer_batch_to_device | 0.0022286 | 35 | 0.078 | 0.078208 |
[Strategy]SingleDeviceStrategy.batch_to_device | 0.0022286 | 35 | 0.078 | 0.078208 |
[LightningModule]LagLlamaLightningModule.on_validation_model_train | 0.008 | 2 | 0.016 | 0.016043 |
[Callback]ModelSummary.on_fit_start | 0.015 | 1 | 0.015 | 0.01504 |
[Callback]TQDMProgressBar.on_validation_batch_end | 0.00078947 | 19 | 0.015 | 0.01504 |
[LightningModule]LagLlamaLightningModule.optimizer_zero_grad | 0.0009375 | 16 | 0.015 | 0.01504 |
Apparently the data loader needs 5 seconds for each batch, which is 84% of the full time of the training step.
After some further investigation, we found that the train data loader does the following:
This means a full timeseries gets transformed and then most of the transformed data is not used. This is then done for each item in a batch. We observed ~10 ms for transforming a full timeseries and with a batch size of 512, we get the >5 seconds reported by the profiler.
The order of execution is partly given by the gluonts package. So I am not aware of an obvious solution without addressing it there.
Now my question. Did you face the same issue during your experiments? How can we solve the problem we observe?
#!git clone https://github.com/time-series-foundation-models/lag-llama/
#cd lag-llama
#!pip install -r requirements.txt --quiet
#!huggingface-cli download time-series-foundation-models/Lag-Llama lag-llama.ckpt --local-dir /content/lag-llama
from gluonts.dataset.common import ListDataset
from itertools import islice
from matplotlib import pyplot as plt
import matplotlib.dates as mdates
import torch
from gluonts.evaluation import make_evaluation_predictions, Evaluator
from gluonts.dataset.repository.datasets import get_dataset
from lag_llama.gluon.estimator import LagLlamaEstimator
import json
import pandas as pd
url = '/content/last_data.csv'
data = pd.read_csv(url)
data['date'] = pd.to_datetime(data['date'], format='%Y-%m-%d %H:%M:%S')
data = data[['date', 'target']]
data.resample('H', on='date')['target'].sum().reset_index()
data = data.set_index('date').asfreq('1H')
context_length = 24
prediction_length = 24*7
freq = "1H"
time_series = data['target'].values
start = data.index[0]
train_ds = ListDataset(
[{'target': time_series[:-prediction_length], 'start': start}],
freq=freq
)
test_ds = ListDataset(
[{'target': time_series, 'start': start}],
freq=freq
)
ckpt = torch.load("lag-llama.ckpt", map_location=torch.device('cuda:0'))
estimator_args = ckpt["hyper_parameters"]["model_kwargs"]
estimator = LagLlamaEstimator(
ckpt_path="lag-llama.ckpt",
prediction_length=prediction_length,
context_length=context_length,
# estimator args
input_size=estimator_args["input_size"],
n_layer=estimator_args["n_layer"],
n_embd_per_head=estimator_args["n_embd_per_head"],
n_head=estimator_args["n_head"],
scaling=estimator_args["scaling"],
time_feat=estimator_args["time_feat"],
)
lightning_module = estimator.create_lightning_module()
transformation = estimator.create_transformation()
predictor = estimator.create_predictor(transformation, lightning_module)
forecast_it, ts_it = make_evaluation_predictions(
dataset=train_ds,
predictor=predictor,
)
forecasts = list(forecast_it)
tss = list(ts_it)
evaluator = Evaluator()
agg_metrics, ts_metrics = evaluator(iter(tss), iter(forecasts))
print(json.dumps(agg_metrics, indent=4))
The results were so bizarre, I couldn't believe it. After 20-30 days on a project, I developed a forecaster model in just 1 minute that performed better than the last model I developed, which took 3 days with an RTX3090. I'm going to lose my mind, holly shit.
Hi guys, thanks for your effort in this work. I just open this issue to figure out how should I compute the CRPS metric, which is mentioned in your papers. I found that the weighted average quantile loss looks very similar to it. Are they same on values? If not , how could I compute the CRPS? Thanks again.
Hi, thank you guys for your outstanding work. And I would like to first reproduce the experiment listed in your paper and then finetune the model on my own dataset. Could you please help to provide the tutorial of how to reproduce the experiment results? It would be very helpful to us I think. Thanks again for your effort.
When I run the fine-tuning code on my dataset I get this error:
TypeError: model
must be a LightningModule
or torch._dynamo.OptimizedModule
, got LagLlamaLightningModule
But the zero-shot works perfectly.
Encountered an incompatibility issue with pandas==2.2.0 where using to_offset('Q') raises an exception for "invalid frequency". This functionality works as expected in pandas==2.1.4. Suggesting an update to the requirements to specify the pandas version to ensure compatibility.
Set up a Docker container with dependencies specified in the requirements file, which defaults to using pandas==2.2.0.
Execute the following Python code:
from pandas.tseries.frequencies import to_offset
to_offset('Q')
Expected Behavior:
The to_offset('Q') call should correctly interpret 'Q' as a quarterly frequency without raising an exception.
When running with pandas==2.2.0, the code raises the following exception:
raise Exception("invalid frequency")
Hi, thanks for your great work.
I have a question about the suggested values for context_length, which seem to be powers of 2. Is there a specific reason for this choice, or could we use other values for context_length that are not powers of 2?
Hello, thank you for sharing the initial code. I aim to replicate the experiment outlined in your paper before proceeding to fine-tune the model with additional datasets. Could you kindly assist me in accessing a tutorial on reproducing the experiment results? Currently, the model seems to lack convergence, so any guidance you could provide would be greatly appreciated.
Once again, thank you for your efforts.
best,
koublal
I have tried the zero-shot prediction on my own dataset, which is related to Healthcare. However, I am not satisfied enough with the performance and perform fine-tuning.
Then things get weird as the performance drops a lot compared to zero-shot. Is this because of one(or both) of the following?
For cases that there is only 1 time series available and needed to be predicted, should one only fine-tune the outer layers? Or maybe prepare more related domain time series(in my case Healthcare) to perform the fine-tuning?
Any suggestion would be helpful, thanks a lot!
import numpy as np
from gluonts.dataset.pandas import PandasDataset
url = (
"/content/Historical Product Demand.csv"
)
df = pd.read_csv(url, index_col=0, parse_dates=True)
# remove_ind = np.random.choice(np.arange(df.shape[0]), size=100, replace=False)
# mask = [False if i in remove_ind else True for i in range(df.shape[0])]
# df_missing_val = df.loc[mask, :] # dataframe with 100 rows removed from df
def change_to_int(val):
try:
return int(val.strip('()')) # Assuming the value is like "(1234)"
except Exception as e:
return 0
# Convert date column to datetime
df['Date'] = pd.to_datetime(df['Date'])
# Apply the function to convert Order_Demand to int
df['Order_Demand'] = df['Order_Demand'].apply(change_to_int)
df['item_id'] = df.index
df['target']= df['Order_Demand']
df.index = df.Date
df = df[['item_id',"target"]]
#Drop duplicate entiries for date
df_missing_val = df[~df.index.duplicated(keep='first')]
df_missing_val=df_missing_val.loc[pd.notnull(df_missing_val.index)]
df
# Get the max end date
freq="1D"
max_end = max(df.groupby("item_id").apply(lambda _df: _df.index[-1]))
print(f"{max_end=}")
dfs_dict = {}
for item_id, gdf in df_missing_val.groupby("item_id"):
# Get the full (regular) date range
# print(f"{gdf.index[0]=}")
new_index = pd.date_range(gdf.index[0], end=max_end, freq=freq)
# Reindex the dataframe
dfs_dict[item_id] = gdf.reindex(new_index).drop("item_id", axis=1)
dfs_dict[item_id].target.fillna(100,inplace=True)
# Conver the columns to float32 for lag-llama
for col in dfs_dict[item_id]:
# Check if column is not of string type
if dfs_dict[item_id][col].dtype != 'object' and pd.api.types.is_string_dtype(dfs_dict[item_id][col]) == False:
dfs_dict[item_id][col] = dfs_dict[item_id][col].astype('float32')
# Create a PandasDataset
ds = PandasDataset(dfs_dict, target="target")
But I am getting the below error when predicting
131 ) -> Iterator:
--> 132 for data_entry in data_it:
133 try:
134 yield self.map_transform(data_entry.copy(), is_train)
[/usr/local/lib/python3.10/dist-packages/gluonts/transform/_base.py](https://localhost:8080/#) in __call__(self, data_it, is_train)
130 self, data_it: Iterable[DataEntry], is_train: bool
131 ) -> Iterator:
--> 132 for data_entry in data_it:
133 try:
134 yield self.map_transform(data_entry.copy(), is_train)
[/usr/local/lib/python3.10/dist-packages/gluonts/dataset/split.py](https://localhost:8080/#) in __iter__(self)
412
413 def __iter__(self):
--> 414 for input, _label in self.test_data:
415 yield input
416
[/usr/local/lib/python3.10/dist-packages/gluonts/dataset/split.py](https://localhost:8080/#) in __iter__(self)
384
385 def __iter__(self) -> Generator[Tuple[DataEntry, DataEntry], None, None]:
--> 386 yield from self.splitter.generate_test_pairs(
387 dataset=self.dataset,
388 prediction_length=self.prediction_length,
[/usr/local/lib/python3.10/dist-packages/gluonts/dataset/split.py](https://localhost:8080/#) in generate_test_pairs(self, dataset, prediction_length, windows, distance, max_history)
249 for window in range(windows):
250 offset = window * distance
--> 251 test = self.test_pair(
252 entry, prediction_length=prediction_length, offset=offset
253 )
[/usr/local/lib/python3.10/dist-packages/gluonts/dataset/split.py](https://localhost:8080/#) in test_pair(self, entry, prediction_length, offset)
299 label_slice = slice(offset_, None)
300 return (
--> 301 slice_data_entry(
302 entry, input_slice, prediction_length=prediction_length
303 ),
[/usr/local/lib/python3.10/dist-packages/gluonts/dataset/split.py](https://localhost:8080/#) in slice_data_entry(entry, slice_, prediction_length)
161 entry: DataEntry, slice_: slice, prediction_length: int = 0
162 ) -> DataEntry:
--> 163 slice_ = to_positive_slice(
164 to_integer_slice(slice_, entry[FieldName.START]),
165 entry[FieldName.TARGET].shape[-1],
[/usr/local/lib/python3.10/dist-packages/gluonts/dataset/split.py](https://localhost:8080/#) in to_positive_slice(slice_, length)
118 if stop is not None and stop < 0:
119 stop += length
--> 120 assert stop >= 0
121 return slice(start, stop, slice_.step)
122
AssertionError:```
Hi! and thankyou for your excellent contribution to the world of time series!
I am currently using lag llama for finetuning, and was wondering if you had any rules of thumb for fine tuning yet?
I have read that transformers generally require many epochs, and noticed your early stopping patience is fifty. Does this mean we should generally train for many epochs? Or was that early stopping patience set on a very small dataset?
For context, my dataset has about 3 years with of price/energy demand/air temp/solar output data at 5 minute intervals. I have set a long context length to try and capture seasonal effects. Wondering how many epochs I should train for. The base foundation model did not work very on my data
Thanks for releasing this foundation model! I wonder if you have code that demonstrates how to finetune Lag-Llama with custom training data? Thanks!
First of thank you for your article and for sharing results
I'am trying to inference model on CPU only device.
And I modified model load code like below
ckpt = torch.load("lag-llama.ckpt", map_location=torch.device('cpu'))
But when execute lightning module still getting Error
lightning_module = estimator.create_lightning_module()
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
Hi !
Thank you for this foundation model, I've been trying to finetune it on a dataset of 10K timeseries but each timeseries only has 100 values (1min frequency and values ranging from 100 to -100). The timeseries are not following each other, so I can't append them.
Using your finetuning notebook, results are not good at all currently. What I'm interested in is predicting the last 20 values for each timeseries I feed the model so the context size should always be 80 + 20.
If I'm not mistaken, currently the finetuning script takes a random context inside each timeseries of the dataset right ?
Do you think my dataset is something the model can be finetuned on or should I build a model from scratch for this task ?
Thank you very much for any insights :)
(tf2) li@lieMacBook-Pro-M1 lag-llama % python run.py
Traceback (most recent call last):
File "/Users/liulan/Downloads/常用脚本/lag-llama/run.py", line 30, in
from gluonts.evaluation import Evaluator, make_evaluation_predictions
ModuleNotFoundError: No module named 'gluonts'
I'm trying to fine-tune the model using my own data. When using the fine-tuned model to make predictions, I find that lag-llama gives all zero value predictions, which does not reproduce the fine-tuning effect in the demo. I followed the following steps:
Call from_long_dataframe to convert the data from the dataframe format to the standard format.
The data contains 7000 second-level data points. I use the first 4000 points for fine-tuning training, and the remaining 3000 data points as model input. The task is to predict the last 60 data points.
The prediction result is shown in the following figure. The result provided by lag-llama is 60 zero values.
What is the cause of this, or Which step did I do wrong?
I am trying to use a dataset with lots of timeseries data as specified in the:
https://ts.gluon.ai/stable/tutorials/data_manipulation/pandasdataframes.html Usecase number 3.
I am unsure on how to use this pandas dataset for this model.
Even when using a single timeseries data as mentioned in the another issue, I am unable to get the forecasts and it shows assertion error: (then its blank)
Hello, it's Arthur,
Thank you for your great work.
I would like to ask whether it is possible to get the specific lag indices you use during the pre-training or zero-shot phases.
In the Colab tutorial notebook, it is indicated that the context length is set to 32, and the maximum potential lag index could be 1092. However, the exact indices used to tokenize the 32 historical time points remain unclear to me.
Do you employ all of the 1092 lags, or is there a specific subset that is used?
Thank you!
Hello,
Thanks for sharing your model.
Could you please provide an example of how to use this outstanding model on a . CSV dataset containing time-series data?
First, I just wanted to say this project is very exciting and I plan to follow along with as it begins to mature, so thank you!
My request would be to have a comparison of Lag-Llama and Meta's Prophet. I know these are different approaches to time series forecasting, but Prophet is pretty widely used in the industry now, so I think potential users will be attracted to Lag-Llama if they see quantifiable comparisons.
Ideally this comparison would include tests from both tools against different datasets/sizes to show accuracy and speed of each.
Good Morning, and thanks for the open source code. I am trying to use the code for a time series classification task. Do you have some hint on what do I have to change?
Thanks for the answer
Hello. How do I freeze the predictions? I have a different prediction every time.
The complete code on Colab:
Using data from Case 1 in GluonTS examples:Hi, I wanted to fine tune the model on my own dataset however with my own custom loss. Could you give an example on how to do that?
Thanks
Why repeat here? The results generated after repeat (i.e. the parameters of the distribution) should be the same, right? What is the significance of this?
lag-llama/lag_llama/gluon/lightning_module.py
Lines 221 to 226 in 9965e30
ckpt = torch.load("lag-llama.ckpt", map_location=torch.device('mps'))
estimator_args = ckpt["hyper_parameters"]["model_kwargs"]
code above is ok but then run following code. occur a error:
estimator = LagLlamaEstimator(
ckpt_path="lag-llama.ckpt",
prediction_length=prediction_length,
context_length=context_length,
# estimator args
input_size=estimator_args["input_size"],
n_layer=estimator_args["n_layer"],
n_embd_per_head=estimator_args["n_embd_per_head"],
n_head=estimator_args["n_head"],
scaling=estimator_args["scaling"],
time_feat=estimator_args["time_feat"],
)
lightning_module = estimator.create_lightning_module()
transformation = estimator.create_transformation()
predictor = estimator.create_predictor(transformation, lightning_module)
`RuntimeError Traceback (most recent call last)
Cell In[31], line 15
1 estimator = LagLlamaEstimator(
2 ckpt_path="lag-llama.ckpt",
3 prediction_length=prediction_length,
(...)
12 time_feat=estimator_args["time_feat"],
13 )
---> 15 lightning_module = estimator.create_lightning_module()
16 transformation = estimator.create_transformation()
17 predictor = estimator.create_predictor(transformation, lightning_module)
File ~/private/time-series-analysis/lag-llama/lag_llama/gluon/estimator.py:280, in LagLlamaEstimator.create_lightning_module(self, use_kv_cache)
264 model_kwargs = {
265 "input_size": self.input_size,
266 "context_length": self.context_length,
(...)
277 "dropout": self.dropout,
278 }
279 if self.ckpt_path is not None:
--> 280 return LagLlamaLightningModule.load_from_checkpoint(
281 checkpoint_path=self.ckpt_path,
282 loss=self.loss,
283 lr=self.lr,
...
262 'to map your storages to the CPU.')
263 device_count = torch.cuda.device_count()
264 if device >= device_count:
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.`
and i modify code still error
ckpt = torch.load("lag-llama.ckpt", map_location=torch.device('cpu'))
estimator_args = ckpt["hyper_parameters"]["model_kwargs"]
RuntimeError Traceback (most recent call last)
Cell In[33], line 15
1 estimator = LagLlamaEstimator(
2 ckpt_path="lag-llama.ckpt",
3 prediction_length=prediction_length,
(...)
12 time_feat=estimator_args["time_feat"],
13 )
---> 15 lightning_module = estimator.create_lightning_module()
16 transformation = estimator.create_transformation()
17 predictor = estimator.create_predictor(transformation, lightning_module)
File ~/private/time-series-analysis/lag-llama/lag_llama/gluon/estimator.py:280, in LagLlamaEstimator.create_lightning_module(self, use_kv_cache)
264 model_kwargs = {
265 "input_size": self.input_size,
266 "context_length": self.context_length,
(...)
277 "dropout": self.dropout,
278 }
279 if self.ckpt_path is not None:
--> 280 return LagLlamaLightningModule.load_from_checkpoint(
281 checkpoint_path=self.ckpt_path,
282 loss=self.loss,
283 lr=self.lr,
...
262 'to map your storages to the CPU.')
263 device_count = torch.cuda.device_count()
264 if device >= device_count:
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
Unable to make future predictions for future timesteps beyond the validation window where there are no known target values. Is that something this model is capable of?
Can we use this model for forecasting on our custom data as of now?
Hi,
Why you did not consider holidays in the preprocessing?
Best regards
File ~/miniconda3/envs/env/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py:580, in Trainer._fit_impl(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
573 ckpt_path = self._checkpoint_connector._select_ckpt_path(
574 self.state.fn,
575 ckpt_path,
...
(...)
137 * self.dummy_value
138 )
KeyError: 'observed_values'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.