[Kst] [kst] [Bug 352367] New: vectors too short. last frame not plotted

[Michael Brewer]

https://bugs.kde.org/show_bug.cgi?id=352367

Bug ID: 352367
Summary: vectors too short. last frame not plotted
Product: kst
Version: 2.0.8
Platform: Ubuntu Packages
OS: Linux
Status: UNCONFIRMED
Severity: major
Priority: NOR
Component: plotting
Assignee: ***@kde.org
Reporter: ***@astro.umass.edu

I have a set of dirfiles containing 600 frames with 200 samples per frame
120000 samples total.
I open the files with Update Type set to No Update. When I create a plot with
Read to End set the INDEX field goes from 0 to 599 instead of 0 to 600 as it
should. Checking View Vectors confirms that the vectors are being cut off at
119801 samples. This is 599 frames plus one sample.


Reproducible: Always

Steps to Reproduce:
1. Create a dirfile with an integer number of frames
2.Make a plot and check the last INDEX value
3. Check View Vectors to confirm that crated vector is truncated

Actual Results:
Vectors contain one less frame than what is present in dirfile

Expected Results:
Create vectors using all data or at least all complete frames present in
dirfile

[Michael Brewer]

https://bugs.kde.org/show_bug.cgi?id=352367

Michael Brewer <***@astro.umass.edu> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@astro.umass.edu

[via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=352367

***@astro.utoronto.ca changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@astro.utoronto.
| |ca

--- Comment #1 from ***@astro.utoronto.ca ---
This is, surprisingly, by design.

We address data in a file by its index. We often plot data in a file against a
field called INDEX, which is always 1 sample per frame.

kst assumes that INDEX is coincident in time with the first sample in a frame.
This clearly makes sense for the first sample of a file, but what to do at the
end of a file is less clear.

The only alternative to this that I can think of is to produce new INDEX-like
pseudo-fields which are at the same rate as any given field. This is
complicated by the fact that different fields may have data at different rates,
so there would need to be one for every SPF in the file. Perhaps fully
automatic?

We can think about this some more.

[Michael Brewer via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=352367

--- Comment #2 from Michael Brewer <***@astro.umass.edu> ---
Following this logic, if my data set consisted of less than 201 samples, KST
would display nothing. In my opinion, the INDEX vector should always extend to
one point beyond the number of frames (or partial frames) present in the data
set to capture all of the data that the user may be interested in. It is like
the X axis on a plot which should extend slightly past the end of the data.
Another way to look at this is that each frame represents a unit of time (one
second in my case). If I have 10 seconds of data, then the INDEX field shoold
extend from 0 seconds to 10 seconds, not from 0 seconds to 9 seconds.

In fact, my data set contains fields with different sample rates. Most of the
data is sampled at 200 Hz. But some of it is sampled at 10 Hz and some at 1 Hz.
Extending the INDEX vector by 1 would capture all of this data.

[via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=352367

Post by Michael Brewer
https://bugs.kde.org/show_bug.cgi?id=352367
Following this logic, if my data set consisted of less than 201 samples, KST
would display nothing.

Right: Imagine you want to plot a field which is 100 samples per frame vs
something that is 1 samples per frame. And you want to plot one frame. Maybe
because your datasource is only one frame long. What should you do?

There is clearly no general solution to this problem.

Post by Michael Brewer
In my opinion, the INDEX vector should always extend
to one point beyond the number of frames (or partial frames) present in the
data set to capture all of the data that the user may be interested in.

INDEX is nothing special: it is something provided by some data sources. It
would be just the same if we were talking about any field sampled once per
frame; say.... "Time". So all vectors from a data source would need to do
this. But then we would need to know to ignore the last extrapolated sample
for everything other than interpolating.

Post by Michael Brewer
It
is like the X axis on a plot which should extend slightly past the end of
the data. Another way to look at this is that each frame represents a unit
of time (one second in my case). If I have 10 seconds of data, then the
INDEX field shoold extend from 0 seconds to 10 seconds, not from 0 seconds
to 9 seconds.

Formally, Index goes from 0 to 10 - (1/sample rate), but it needs to be an
integer.

Post by Michael Brewer
In fact, my data set contains fields with different sample rates. Most of
the data is sampled at 200 Hz. But some of it is sampled at 10 Hz and some
at 1 Hz. Extending the INDEX vector by 1 would capture all of this data.

It is a tricky problem to get right everywhere, but I will mull upon it
further (and not close the bug...)

[Michael Brewer via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=352367

--- Comment #4 from Michael Brewer <***@astro.umass.edu> ---
From your previous comment, I got the impression that INDEX was something
specific to the plot. If INDEX is in fact just another 1 sample per frame
vector, then I agree that it should be the same as all the other 1 sample per
frame vectors.

*******
Right: Imagine you want to plot a field which is 100 samples per frame vs
something that is 1 samples per frame. And you want to plot one frame. Maybe
because your datasource is only one frame long. What should you do?
*******

Currently, this question is moot since it is not possible to plot one frame of
data in KST.
But that is my point. If the user has only one frame of data, there should be a
way for them to look at that data. In the example that you give, there is only
one X-Y pair to plot. In order to plot this, the plot itself needs to extend
from 0 to 1. In general, the plot should always extend from 0 to nframes. This
is one more than the INDEX array that extends from 0 to nframes - 1. Since this
is just a matter of setting the boundaries of the X axis of the plot, it seems
straight forward to me. Perhaps I am missing something though.

[Michael Brewer via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=352367

--- Comment #5 from Michael Brewer <***@astro.umass.edu> ---
(In reply to Michael Brewer from comment #4)

Post by Michael Brewer via KDE Bugzilla
From your previous comment, I got the impression that INDEX was something
specific to the plot. If INDEX is in fact just another 1 sample per frame
vector, then I agree that it should be the same as all the other 1 sample
per frame vectors.
*******
Right: Imagine you want to plot a field which is 100 samples per frame vs
something that is 1 samples per frame. And you want to plot one frame. Maybe
because your datasource is only one frame long. What should you do?
*******
Currently, this question is moot since it is not possible to plot one frame
of data in KST.
But that is my point. If the user has only one frame of data, there should
be a way for them to look at that data. In the example that you give, there
is only one X-Y pair to plot. In order to plot this, the plot itself needs
to extend from 0 to 1. In general, the plot should always extend from 0 to
nframes. This is one more than the INDEX array that extends from 0 to
nframes - 1. Since this is just a matter of setting the boundaries of the X
axis of the plot, it seems straight forward to me. Perhaps I am missing
something though.

BTW, in your example if the user has two fields of data at 100 samples per
frame that they want to plot using one field as X and the other as Y, then
there are 100 X-Y pairs to plot. Does it make sense that KST should do nothing?