BufferPicking

Buffer Picking

Buffer Picking class allows programmers to render 3d objects into a 2d buffer(texture), and then read pixels values from the buffer. The technique is frequently used for pixel-accurate mouse picking in the 3d scene.

There are two predefined buffers:

• "backbuffer": this is the buffer of what you see on the screen.
• "overlay": this is an additional render pass for objects on top of 3d scene.

One can also create any custom picking buffers.

Picking from "backbuffer"

If one just wants to get pixel color that is displayed on the screen, use following

NPL.load("(gl)script/ide/System/Scene/BufferPicking.lua");
local BufferPicking = commonlib.gettable("System.Scene.BufferPicking");
local result = BufferPicking:Pick(nil, nil, 2, 2);
echo(result);
echo({System.Core.Color.DWORD_TO_RGBA(result[1] or 0)});

Picking from "overlay" objects

If one wants to pick from overlay objects, use OverlayPicking.lua

All manipulators in paracraft are rendered using Overlay objects, which can be picked via OverlayPicking.

Picking from custom buffers

One can also create a custom buffer and render any 3d objects on demand.

NPL.load("(gl)script/ide/System/Scene/BufferPicking.lua");
local MyPickBuffer = commonlib.inherit(commonlib.gettable("System.Scene.BufferPicking"), commonlib.gettable("Tests.MyPickBuffer"));
MyPickBuffer:Property("Name", "MyCustomBuffer"); 
-- only called when Pick function is called 
function MyPickBuffer:paintEvent(painter)
	self:SetColorAndName(painter, "#ff0000");
	painter:PushMatrix();
	local obj = ParaScene.GetPlayer();
	local x, y, z = obj:GetPosition();
	local vOrigin = self:GetRenderOrigin();
	x, y, z = x - vOrigin[1], y - vOrigin[2], z - vOrigin[3];
	painter:TranslateMatrix(x,y,z);
	painter:DrawSceneObject(obj, 0)
	painter:PopMatrix();
end
MyPickBuffer:InitSingleton();

-- for debugging purposes, we will show the picking buffer into the gui. 
MyPickBuffer:DebugShow("_lt", 10, 10, 128, 128)
	
-- test picking here
commonlib.Timer:new({callbackFunc = function(timer)
	-- always redraw (force paintEvent to be invoked)
	MyPickBuffer:SetDirty(true);
	-- pick at the current mouse position
	echo(MyPickBuffer:Pick(nil, nil, 2, 2));
end}):Change(0, 1000)

You may notice this line in the paintEvent of above code

local vOrigin = self:GetRenderOrigin();
x, y, z = x - vOrigin[1], y - vOrigin[2], z - vOrigin[3];

World position has double precision which is usually far away from the camera origin. When rendering 3d objects, object's world position must subtract this vector, so that all objects are close to camera before we do any matrix4 transforms.

References

• script/ide/System/Scene/BufferPicking.lua
• script/ide/System/test/test_bufferpicking.lua
• script/ide/System/Scene/Overlays/OverlayPicking.lua