Search code examples
matlab3d2dprojection

MATLAB: How to create 2d pixel map of data stored in 3d patch object's FaceVertexCData based on camera view?

Minimum viable example:

% Define a simple 6 sided cube with dimensions 0-1 in X, Y, and Z
vertices = [0,0,1;  % 1 Front-Top-Left (FTL)
            1,0,1;  % 2 Front-Top-Right (FTR)
            1,0,0;  % 3 Front-Bottom-Right (FBoR)
            0,0,0;  % 4 Front-Bottom-Left (FBoL)
            0,1,1;  % 5 Back-Top-Left (BaTL)
            1,1,1;  % 6 Back-Top-Right (BaTR)
            1,1,0;  % 7 Back-Bottom-Right (BaBoR)
            0,1,0]; % 8 Back-Bottom-Left (BaBoL)
faces = [1,2,3,4;  % 1 Front
         2,6,7,3;  % 2 Right
         6,5,8,7;  % 3 Back
         5,1,4,8;  % 4 Left
         1,5,6,2;  % 5 Top
         4,3,7,8]; % 6 Bottom
temps = [5;10;15;20;25;30]; % One temperature per face
% Plot the cube
fig1 = figure;
ax1 = axes(fig1);
patch('Faces',faces,'Vertices',vertices,'FaceColor','flat','EdgeColor','none','FaceVertexCData',temps,'Parent',ax1);
% Rotate to an isometric type of view angle
ax1.View = [-42.5 28];
% Turn off axes
ax1.XAxis.Visible = 'off';
ax1.YAxis.Visible = 'off';
ax1.ZAxis.Visible = 'off';
% Set data aspect ratio to 1 1 1
daspect(ax1,[1 1 1]);

I then need code that will produce a matrix that looks something like this (for the above code):

% -------------------------------------------------------------------------
% Code goes here that should produce a 2D matrix that looks something like:
% (15x15 as example, but would need to be more like 1024x1024 up to 
%  4096x4096 in real world use cases)
%       1  2  3  4  5  6  7  8  9 10 11 12 13 14 15
% 1  % 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
% 2  % 00 00 00 00 00 00 25 00 00 00 00 00 00 00 00 
% 3  % 00 00 00 00 00 25 25 25 00 00 00 00 00 00 00 
% 4  % 00 00 00 00 25 25 25 25 25 00 00 00 00 00 00 
% 5  % 00 00 00 25 25 25 25 25 25 25 00 00 00 00 00 
% 6  % 00 00 20 25 25 25 25 25 25 10 00 00 00 00 00 
% 7  % 00 00 20 20 20 25 25 25 10 10 00 00 00 00 00 
% 8  % 00 00 20 20 20 20 25 10 10 10 00 00 00 00 00 
% 9  % 00 00 20 20 20 20 20 10 10 10 00 00 00 00 00 
% 10 % 00 00 20 20 20 20 20 10 10 10 00 00 00 00 00 
% 11 % 00 00 00 20 20 20 20 10 10 00 00 00 00 00 00 
% 12 % 00 00 00 00 20 20 20 10 00 00 00 00 00 00 00 
% 13 % 00 00 00 00 00 20 20 10 00 00 00 00 00 00 00 
% 14 % 00 00 00 00 00 00 20 00 00 00 00 00 00 00 00 
% 15 % 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Real world use cases would include projection from multiple different angles onto complex 3D geometries containing in excess of 100k polygons. There will also be cases where part of the geometry overlaps other parts of the geometry, and the solution needs to account for only the visible geometry from the specific look angle and ignore all other facets out of sight behind the 'closest to the camera' faces. The current plotting solution plots one patch object per body as children of an hgtransform and there can be hundreds of bodies in a model.

Any help would be appreciated!

Solution

  • Let's assume you have access to the MATLAB figure and that it is in the desired view. The idea is to capture the image displayed in the figure, and map the colors to the initial values using the colormap and its associated clim.

    % Extract colormap limits
h = gca;
lims = h.CLim;

% Extract colormap RGB values and create correspondence vector from lims
map = colormap();
vals = linspace(lims(1),lims(2),size(map,1));

% Get the figure color content from the current view (convert to double
% precision)
F = getframe(fig1);
Fdouble = im2double(F.cdata);

% Map each pixel RGB value in F.cdata to the corresponding value, using the map
% and the corresponding vals
% To do so, for each pixel color (F.cdata(ii,jj,:)) get idx of the closest color in map
out = zeros(size(F.cdata,1),size(F.cdata,2));

for ii = 1:size(F.cdata,1)
    for jj = 1:size(F.cdata,2)
        
        [M,I] = min(sqrt((map(:,1)-Fdouble(ii,jj,1)).^2+(map(:,2)-Fdouble(ii,jj,2)).^2+(map(:,3)-Fdouble(ii,jj,3)).^2));
        % if the color is close enough, get the corresponding value in
        % vals. if not keep 0;

        if M < 0.1

            out(ii,jj) = vals(I);

        end

    end
end

    Output

    The output out is an array containing the temperature values. Funnily enough, the easiest way to represent it is to surf it back to an image. You can see with the datatips that the values are corrrect:

    enter image description here

    Side note #1

    The output is an array with the same resolution than the original figure (meaning you can adjust the resolution of the array slightly by adjusting the size of the actual figure). See the two examples below:

    Lower resolution input enter image description here

    Higher resolution input enter image description here

    Side note #2

    You can get rid of the margin around your plot by having your axes take the whole figure space:

    set(gca,'position',[0 0 1 1],'units','normalized')

    Side note #3

    It appears that, for this specific example, changing the graphics renderer to 'painters' removes the artifacts from the sides of the cube:

    set(gcf,'renderer','painters')

    enter image description here

    Side note #4

    By default, the GraphicsSmoothing option is on in MATLAB's plots. This is not good for what you want to do because it is going to interpolate the colors in the transition zones, leading to colors possibly outside of your colormap palette.

    setting set(gcf,'GraphicsSmoothing','off'); before the use of getframe gives a way cleaner output:

    enter image description here

    Side note #5

    getframe does not allow you to change the size of its output image. You can have a better resolution in your output by saving to an image file in high quality and then reimport it into MATLAB:

    exportgraphics(gca,"highres.tiff","Resolution",600);
F = imread("highres.tiff");
% Convert to doubles
Fdouble = im2double(F);

    enter image description here

    Final code

    set(gca,'position',[0 0 1 1],'units','normalized')

% Apparently the 'painters' renderer works better at eliminating artefacts
% on the patch sides
 set(gcf,'renderer','painters');
% Prevent graphics smoothing to prevent colors that would be out of the
% colormap
 set(gcf,'GraphicsSmoothing','off');

% Extract colormap limits
h = gca;
lims = h.CLim;

% Extract colormap RGB values and create correspondence vector from lims
map = colormap();
vals = linspace(lims(1),lims(2),size(map,1));

exportgraphics(gca,"highres.tiff","Resolution",600);
F = imread("highres.tiff");
% Convert to doubles
Fdouble = im2double(F);   

% Map each pixel RGB value in F.cdata to the corresponding value, using the map
% and the corresponding vals
% To do so, for each pixel color (F.cdata(ii,jj,:)) get idx of the closest color in map
out = zeros(size(Fdouble,1),size(Fdouble,2));
mins_vals = zeros(size(Fdouble,1),size(Fdouble,2));

for ii = 1:size(Fdouble,1)
    for jj = 1:size(Fdouble,2)
        
        [M,I] = min(sqrt((map(:,1)-Fdouble(ii,jj,1)).^2+(map(:,2)-Fdouble(ii,jj,2)).^2+(map(:,3)-Fdouble(ii,jj,3)).^2));
        % if the color is close enough, get the corresponding value in
        % vals. if not keep 0;

        mins_vals(ii,jj) = M;

        if M < 0.1

            out(ii,jj) = vals(I);

        end

    end
end