Skip to content

the process to set graph yaxis min/max based on bitdepth is broken #647

New issue
New issue
Closed
Closed
the process to set graph yaxis min/max based on bitdepth is broken#647
Assignees
ElderOrb
@dericed

Description

@dericed
dericed
opened on Oct 9, 2020

Referencing:

qctools/Source/GUI/Plot.cpp

Lines 457 to 459 in 3b61dd1

m_engine.globalObject().setProperty("bitsPerRawSample", bitsPerRawSample);
m_engine.globalObject().setProperty("two_pow_bitsPerRawSample_minus_one", (1 << bitsPerRawSample) - 1);
m_engine.globalObject().setProperty("sqrt_pow_bitsPerRawSample_2", sqrt(2) * (1 << bitsPerRawSample) / 2);
and

qctools/Source/Core/VideoCore.cpp

Line 15 in a819b59

Item_YMIN, 5, "0", "function() { return two_pow_bitsPerRawSample_minus_one; }", 3, "Y", true,
.

You can make a sample via:

ffmpeg -f lavfi -i testsrc2 -c:v v210 -vf lut=y=val/2 -t 10 test_0-512.mov. The bit depth of this file is 10 so the yaxis should be 0-(2^(bitdepth)-1) = 0...1023, but the max is set to around 420 or so.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions