Kernel dies running everest.standalone.DetrendFITS on short cadence data about everest HOT 13 OPEN

Hi @michaelcarini, sorry for the delay. The error you're getting is expected: Everest is trying to invert a 100,000 x 100,000 matrix when computing the GP and is choking. What I do when running short cadence is to pass the breakpoints=[...] keyword argument to DetrendFITS (or any Everest model). This should be a list of indices (ranging from 0 to the number of cadences in the campaign) at which to divide the light curve. The GP (and PLD) models will be computed separately in each chunk. You can see that for the first 8 campaigns, I default to something like 30 breakpoints for the short cadence light curves.

This isn't ideal since it can introduce artifacts at those indices, but in practice it typically works OK. I've been meaning to switch from george to celerite for computing the GP, as the latter can handle covariance matrices this big with no problem. But I haven't yet had time...

Let me know if this works.

from everest.

from everest.

from everest.

Hmm, with 30 breakpoints I can just run everything on my Macbook. If you have 30 evenly spaced breakpoints, the memory load should be the same as for LC data (since each chunk will have about the same number of cadences as a single LC light curve). Which campaign are you running?

As for the interactive aperture selector, yes, closing the window selects the aperture you chose.

from everest.

from everest.

from everest.

Hi @michaelcarini, I think I know what's going on. Typically when I run the short cadence light curves, I've already processed the long cadence ones, so EVEREST knows to use the GP hyperparameters from the LC run. What's happening is EVEREST is trying to optimize the GP kernel by inverting the full covariance matrix for the SC data, which is guaranteed to give you errors.
What you can do is pass optimize_gp=False to DetrendFITSand manually specify the kernel parameters via the kernel_params keyword, which accepts a tuple of white noise amplitude in flux units, red noise amplitude in flux units, and timescale in days for the Matern-3/2 kernel. The simplest way to set those parameters is to look at what the GP hyperparameters are in the long cadence run. Is this a target that was processed in the LC EVEREST catalog? If so, just grab the FITS file (you can search for it here) and check out the GPWHITE, GPRED, and GPTAU parameters in the header. If the target is not in the EVEREST catalog, you can de-trend it yourself in LC using DetrendFITS and look at the resulting FITS file header.

Sorry this has caused you so much trouble. Let me know how this sounds to you -- I'd be happy to give a shot at this if you send me the target number.

from everest.

from everest.

from everest.

@michaelcarini I sent you an email with the de-trended FITS file. Let me know if that worked and I'll close the issue.

from everest.

from everest.

You should have a local folder named

~/.everest2/k2/c102/229100000/51988/

If not, create it, and place the file hlsp_everest_k2_llc_229151988-c102_kepler_v2.0_sc.fits in there. Then, when you call

star_sc = everest.Everest(229151988, cadence="sc", season=102)

from python, it should be able to access the file. Let me know if this doesn't work.

from everest.

from everest.