WPF Image Pan, Zoom and Scroll with layers on a canvas

Question

I'm hoping someone can help me out here. I'm building a WPF imaging application that takes live images from a camera allowing users to view the image, and subsequently highlight regions of interest (ROI) on that image. Information about the ROIs (width, height, location relative to a point on the image, etc) is then sent back to the camera, in effect telling/training the camera firmware where to look for things like barcodes, text, liquid levels, turns on a screw, etc. on the image). A desired feature is the ability to pan and zoom the image and it's ROIs, as well as scroll when the image is zoomed larger than the viewing area. The StrokeThickness and FontSize of the ROI's need to keep there original scale, but the width and height of the shapes within an ROI need to scale with the image (this is critical to capture exact pixel locations to transmit to the camera). I've got most of this worked out with the exception of scrolling and a few other issues. My two areas of concern are:

1. When I introduce a ScrollViewer I don't get any scroll behavior. As I understand it I need to introduce a LayoutTransform to get the correct ScrollViewer behavior. However when I do that other areas start to break down (e.g. ROIs don't hold their correct position over the image, or the mouse pointer begins to creep away from the selected point on the image when panning, or the left corner of my image bounces to the current mouse position on MouseDown .)

2. I can't quite get the scaling of my ROI's the way I need them. I have this working, but it is not ideal. What I have doesn't retain the exact stroke thickness, and I haven't looked into ignoring scale on the textblocks. Hopefully you'll see what I'm doing in the code samples.

I'm sure my issue has something to do with my lack of understanding of Transforms and their relationship to the WPF layout system. Hopefully a rendition of the code that exhibits what I've accomplished so far will help (see below).

FYI, if Adorners are the suggestion, that may not work in my scenario because I could end up with more adorners than are supported (rumor 144 adorners is when things start breaking down).

First off, below is a screenshot showing an image with to ROI's (text and a shape). The rectangle, ellipse and text need to follow the area on the image in scale and rotation, but not they shouldn't scale in thickness or fontsize.

Here's the XAML that is showing the above image, along with a Slider for zooming (mousewheel zoom will come later)

<Window x:Class="PanZoomStackOverflow.MainWindow"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    mc:Ignorable="d"
    Title="MainWindow" Height="768" Width="1024">

<DockPanel>
  <Slider x:Name="_ImageZoomSlider" DockPanel.Dock="Bottom"
          Value="2"
          HorizontalAlignment="Center" Margin="6,0,0,0" 
          Width="143" Minimum=".5" Maximum="20" SmallChange=".1" 
          LargeChange=".2" TickFrequency="2" 
          TickPlacement="BottomRight" Padding="0" Height="23"/>

  <!-- This resides in a user control in my solution -->
  <Grid x:Name="LayoutRoot">
    <ScrollViewer Name="border" HorizontalScrollBarVisibility="Auto" 
                  VerticalScrollBarVisibility="Auto">
      <Grid x:Name="_ImageDisplayGrid">
        <Image x:Name="_DisplayImage" Margin="2" Stretch="None"
               Source="Untitled.bmp"
               RenderTransformOrigin ="0.5,0.5"
               RenderOptions.BitmapScalingMode="NearestNeighbor"
               MouseLeftButtonDown="ImageScrollArea_MouseLeftButtonDown"
               MouseLeftButtonUp="ImageScrollArea_MouseLeftButtonUp"
               MouseMove="ImageScrollArea_MouseMove">                            
           <Image.LayoutTransform>
             <TransformGroup>
               <ScaleTransform />
               <TranslateTransform />
             </TransformGroup>
           </Image.LayoutTransform>
         </Image>
         <AdornerDecorator> <!-- Using this Adorner Decorator for Move, Resize and Rotation and feedback adornernments -->
           <Canvas x:Name="_ROICollectionCanvas"
                   Width="{Binding ElementName=_DisplayImage, Path=ActualWidth, Mode=OneWay}"
                   Height="{Binding ElementName=_DisplayImage, Path=ActualHeight, Mode=OneWay}"
                   Margin="{Binding ElementName=_DisplayImage, Path=Margin, Mode=OneWay}">

             <!-- This is a user control in my solution -->
             <Grid IsHitTestVisible="False" Canvas.Left="138" Canvas.Top="58" Height="25" Width="186">
               <TextBlock Text="Rectangle ROI" HorizontalAlignment="Center" VerticalAlignment="Top" 
                          Foreground="Orange" FontWeight="Bold" Margin="0,-15,0,0"/>
                 <Rectangle StrokeThickness="2" Stroke="Orange"/>
             </Grid>

             <!-- This is a user control in my solution -->
             <Grid IsHitTestVisible="False" Canvas.Left="176" Canvas.Top="154" Height="65" Width="69">
               <TextBlock Text="Ellipse ROI" HorizontalAlignment="Center" VerticalAlignment="Top" 
                          Foreground="Orange" FontWeight="Bold" Margin="0,-15,0,0"/>
               <Ellipse StrokeThickness="2" Stroke="Orange"/>
             </Grid>
           </Canvas>
         </AdornerDecorator>
       </Grid>
     </ScrollViewer>
  </Grid>
</DockPanel>

Here's the C# that manages pan and zoom.

public partial class MainWindow : Window
{
private Point origin;
private Point start;
private Slider _slider;

public MainWindow()
{
    this.InitializeComponent();

    //Setup a transform group that we'll use to manage panning of the image area
    TransformGroup group = new TransformGroup();
    ScaleTransform st = new ScaleTransform();
    group.Children.Add(st);
    TranslateTransform tt = new TranslateTransform();
    group.Children.Add(tt);
    //Wire up the slider to the image for zooming
    _slider = _ImageZoomSlider;
    _slider.ValueChanged += _ImageZoomSlider_ValueChanged;
    st.ScaleX = _slider.Value;
    st.ScaleY = _slider.Value;
    //_ImageScrollArea.RenderTransformOrigin = new Point(0.5, 0.5);
    //_ImageScrollArea.LayoutTransform = group;
    _DisplayImage.RenderTransformOrigin = new Point(0.5, 0.5);
    _DisplayImage.RenderTransform = group;
    _ROICollectionCanvas.RenderTransformOrigin = new Point(0.5, 0.5);
    _ROICollectionCanvas.RenderTransform = group;
}

//Captures the mouse to prepare for panning the scrollable image area
private void ImageScrollArea_MouseLeftButtonUp(object sender, MouseButtonEventArgs e)
{
    _DisplayImage.ReleaseMouseCapture();
}

//Moves/Pans the scrollable image area  assuming mouse is captured.
private void ImageScrollArea_MouseMove(object sender, MouseEventArgs e)
{
    if (!_DisplayImage.IsMouseCaptured) return;

    var tt = (TranslateTransform)((TransformGroup)_DisplayImage.RenderTransform).Children.First(tr => tr is TranslateTransform);

    Vector v = start - e.GetPosition(border);
    tt.X = origin.X - v.X;
    tt.Y = origin.Y - v.Y;
}

//Cleanup for Move/Pan when mouse is released
private void ImageScrollArea_MouseLeftButtonDown(object sender, MouseButtonEventArgs e)
{
    _DisplayImage.CaptureMouse();
    var tt = (TranslateTransform)((TransformGroup)_DisplayImage.RenderTransform).Children.First(tr => tr is TranslateTransform);
    start = e.GetPosition(border);
    origin = new Point(tt.X, tt.Y);
}

//Zoom according to the slider changes
private void _ImageZoomSlider_ValueChanged(object sender, RoutedPropertyChangedEventArgs<double> e)
{
    //Panel panel = _ImageScrollArea;
    Image panel = _DisplayImage;

    //Set the scale coordinates on the ScaleTransform from the slider
    ScaleTransform transform = (ScaleTransform)((TransformGroup)panel.RenderTransform).Children.First(tr => tr is ScaleTransform);
    transform.ScaleX = _slider.Value;
    transform.ScaleY = _slider.Value;


    //Set the zoom (this will affect rotate too) origin to the center of the panel
    panel.RenderTransformOrigin = new Point(0.5, 0.5);

    foreach (UIElement child in _ROICollectionCanvas.Children)
    {
        //Assume all shapes are contained in a panel
        Panel childPanel = child as Panel;

        var x = childPanel.Children;

        //Shape width and heigh should scale, but not StrokeThickness
        foreach (var shape in childPanel.Children.OfType<Shape>())
        {
            if (shape.Tag == null)
            {
                //Hack: This is be a property on a usercontrol in my solution
                shape.Tag = shape.StrokeThickness;
            }
            double orignalStrokeThickness = (double)shape.Tag;

            //Attempt to keep the underlying shape border/stroke from thickening as well
            double newThickness = shape.StrokeThickness - (orignalStrokeThickness / transform.ScaleX);

            shape.StrokeThickness -= newThickness;
        }
    }
}
}

The code should work in a .NET 4.0 or 4.5 project and solution, assuming no cut/paste errors.

Any thoughts? Suggestions are welcome.

Answer 1

Ok. This is my take on what you described.

It looks like this:

• Since I'm not applying any RenderTransforms, I get the desired Scrollbar / ScrollViewer functionality.
• MVVM, which is THE way to go in WPF. UI and data are independent thus the DataItems only have double and int properties for X,Y, Width,Height, etc that you can use for whatever purposes or even store them in a Database.
• I added the whole stuff inside a Thumb to handle the panning. You will still need to do something about the Panning that occurs when you are dragging / resizing a ROI via the ResizerControl. I guess you can check for Mouse.DirectlyOver or something.
• I actually used a ListBox to handle the ROIs so that you may have 1 selected ROI at any given time. This toggles the Resizing Functionality. So that if you click on a ROI, you will get the resizer visible.
• The Scaling is handled at the ViewModel level, thus eliminating the need for custom Panels or stuff like that (though @Clemens' solution is nice as well)
• I'm using an Enum and some DataTriggers to define the Shapes. See the DataTemplate DataType={x:Type local:ROI} part.

• WPF Rocks. Just Copy and paste my code in a File -> New Project -> WPF Application and see the results for yourself.

  <Window x:Class="MiscSamples.PanZoomStackOverflow_MVVM"
          xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
          xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
          xmlns:local="clr-namespace:MiscSamples"
          Title="PanZoomStackOverflow_MVVM" Height="300" Width="300">
     <Window.Resources>
      <DataTemplate DataType="{x:Type local:ROI}">
          <Grid Background="#01FFFFFF">
              <Path x:Name="Path" StrokeThickness="2" Stroke="Black"
                    Stretch="Fill"/>
              <local:ResizerControl Visibility="Collapsed" Background="#30FFFFFF"
                                    X="{Binding X}" Y="{Binding Y}"
                                    ItemWidth="{Binding Width}"
                                    ItemHeight="{Binding Height}"
                                    x:Name="Resizer"/>
          </Grid>
          <DataTemplate.Triggers>
              <DataTrigger Binding="{Binding IsSelected, RelativeSource={RelativeSource AncestorType=ListBoxItem}}" Value="True">
                  <Setter TargetName="Resizer" Property="Visibility" Value="Visible"/>
              </DataTrigger>
              <DataTrigger Binding="{Binding Shape}" Value="{x:Static local:Shapes.Square}">
                  <Setter TargetName="Path" Property="Data">
                      <Setter.Value>
                          <RectangleGeometry Rect="0,0,10,10"/>
                      </Setter.Value>
                  </Setter>
              </DataTrigger>
  
              <DataTrigger Binding="{Binding Shape}" Value="{x:Static local:Shapes.Round}">
                  <Setter TargetName="Path" Property="Data">
                      <Setter.Value>
                          <EllipseGeometry RadiusX="10" RadiusY="10"/>
                      </Setter.Value>
                  </Setter>
              </DataTrigger>
          </DataTemplate.Triggers>
      </DataTemplate>
  
      <Style TargetType="ListBox" x:Key="ROIListBoxStyle">
          <Setter Property="ItemsPanel">
              <Setter.Value>
                  <ItemsPanelTemplate>
                      <Canvas/>
                  </ItemsPanelTemplate>
              </Setter.Value>
          </Setter>
          <Setter Property="Template">
              <Setter.Value>
                  <ControlTemplate>
                      <ItemsPresenter/>
                  </ControlTemplate>
              </Setter.Value>
          </Setter>
      </Style>
      <Style TargetType="ListBoxItem" x:Key="ROIItemStyle">
          <Setter Property="Canvas.Left" Value="{Binding ActualX}"/>
          <Setter Property="Canvas.Top" Value="{Binding ActualY}"/>
          <Setter Property="Height" Value="{Binding ActualHeight}"/>
          <Setter Property="Width" Value="{Binding ActualWidth}"/>
  
          <Setter Property="Template">
              <Setter.Value>
                  <ControlTemplate TargetType="ListBoxItem">
                      <ContentPresenter ContentSource="Content"/>
                  </ControlTemplate>
              </Setter.Value>
          </Setter>
      </Style>
  
  </Window.Resources>
  
  <DockPanel>
      <Slider VerticalAlignment="Center" 
              Maximum="2" Minimum="0" Value="{Binding ScaleFactor}" SmallChange=".1"
              DockPanel.Dock="Bottom"/>
  
      <ScrollViewer VerticalScrollBarVisibility="Visible"
                    HorizontalScrollBarVisibility="Visible" x:Name="scr"
                    ScrollChanged="ScrollChanged">
          <Thumb DragDelta="Thumb_DragDelta">
              <Thumb.Template>
                  <ControlTemplate>
                      <Grid>
                          <Image Source="/Images/Homer.jpg" Stretch="None" x:Name="Img"
                                  VerticalAlignment="Top" HorizontalAlignment="Left">
                              <Image.LayoutTransform>
                                  <TransformGroup>
                                      <ScaleTransform ScaleX="{Binding ScaleFactor}" ScaleY="{Binding ScaleFactor}"/>
                                  </TransformGroup>
                              </Image.LayoutTransform>
                          </Image>
  
                          <ListBox ItemsSource="{Binding ROIs}"
                                   Width="{Binding ActualWidth, ElementName=Img}"
                                   Height="{Binding ActualHeight,ElementName=Img}"
                                   VerticalAlignment="Top" HorizontalAlignment="Left"
                                   Style="{StaticResource ROIListBoxStyle}"
                                   ItemContainerStyle="{StaticResource ROIItemStyle}"/>
                      </Grid>
                  </ControlTemplate>
              </Thumb.Template>
          </Thumb>
      </ScrollViewer>
  </DockPanel>
  

Code Behind:

public partial class PanZoomStackOverflow_MVVM : Window
    {
        public PanZoomViewModel ViewModel { get; set; }

        public PanZoomStackOverflow_MVVM()
        {
            InitializeComponent();
            DataContext = ViewModel = new PanZoomViewModel();

            ViewModel.ROIs.Add(new ROI() {ScaleFactor = ViewModel.ScaleFactor, X = 150, Y = 150, Height = 200, Width = 200, Shape = Shapes.Square});

            ViewModel.ROIs.Add(new ROI() { ScaleFactor = ViewModel.ScaleFactor, X = 50, Y = 230, Height = 102, Width = 300, Shape = Shapes.Round });
        }

        private void Thumb_DragDelta(object sender, DragDeltaEventArgs e)
        {
            //TODO: Detect whether a ROI is being resized / dragged and prevent Panning if so.
            IsPanning = true;
            ViewModel.OffsetX = (ViewModel.OffsetX + (((e.HorizontalChange/10) * -1) * ViewModel.ScaleFactor));
            ViewModel.OffsetY = (ViewModel.OffsetY + (((e.VerticalChange/10) * -1) * ViewModel.ScaleFactor));

            scr.ScrollToVerticalOffset(ViewModel.OffsetY);
            scr.ScrollToHorizontalOffset(ViewModel.OffsetX);

            IsPanning = false;
        }

        private bool IsPanning { get; set; }

        private void ScrollChanged(object sender, ScrollChangedEventArgs e)
        {
            if (!IsPanning)
            {
                ViewModel.OffsetX = e.HorizontalOffset;
                ViewModel.OffsetY = e.VerticalOffset;
            }
        }
    }

Main ViewModel:

public class PanZoomViewModel:PropertyChangedBase
{
    private double _offsetX;
    public double OffsetX
    {
        get { return _offsetX; }
        set
        {
            _offsetX = value;
            OnPropertyChanged("OffsetX");
        }
    }

    private double _offsetY;
    public double OffsetY
    {
        get { return _offsetY; }
        set
        {
            _offsetY = value;
            OnPropertyChanged("OffsetY");
        }
    }

    private double _scaleFactor = 1;
    public double ScaleFactor
    {
        get { return _scaleFactor; }
        set
        {
            _scaleFactor = value;
            OnPropertyChanged("ScaleFactor");
            ROIs.ToList().ForEach(x => x.ScaleFactor = value);
        }
    }

    private ObservableCollection<ROI> _rois;
    public ObservableCollection<ROI> ROIs
    {
        get { return _rois ?? (_rois = new ObservableCollection<ROI>()); }
    }
}

ROI ViewModel:

public class ROI:PropertyChangedBase
{
    private Shapes _shape;
    public Shapes Shape
    {
        get { return _shape; }
        set
        {
            _shape = value;
            OnPropertyChanged("Shape");
        }
    }

    private double _scaleFactor;
    public double ScaleFactor
    {
        get { return _scaleFactor; }
        set
        {
            _scaleFactor = value;
            OnPropertyChanged("ScaleFactor");
            OnPropertyChanged("ActualX");
            OnPropertyChanged("ActualY");
            OnPropertyChanged("ActualHeight");
            OnPropertyChanged("ActualWidth");
        }
    }

    private double _x;
    public double X
    {
        get { return _x; }
        set
        {
            _x = value;
            OnPropertyChanged("X");
            OnPropertyChanged("ActualX");
        }
    }

    private double _y;
    public double Y
    {
        get { return _y; }
        set
        {
            _y = value;
            OnPropertyChanged("Y");
            OnPropertyChanged("ActualY");
        }
    }

    private double _height;
    public double Height
    {
        get { return _height; }
        set
        {
            _height = value;
            OnPropertyChanged("Height");
            OnPropertyChanged("ActualHeight");
        }
    }

    private double _width;
    public double Width
    {
        get { return _width; }
        set
        {
            _width = value;
            OnPropertyChanged("Width");
            OnPropertyChanged("ActualWidth");
        }
    }

    public double ActualX { get { return X*ScaleFactor; }}
    public double ActualY { get { return Y*ScaleFactor; }}
    public double ActualWidth { get { return Width*ScaleFactor; }}
    public double ActualHeight { get { return Height * ScaleFactor; } }
}

Shapes Enum:

public enum Shapes
{
    Round = 1,
    Square = 2,
    AnyOther
}

PropertyChangedBase (MVVM Helper class):

    public class PropertyChangedBase:INotifyPropertyChanged
    {
        public event PropertyChangedEventHandler PropertyChanged;

        protected virtual void OnPropertyChanged(string propertyName)
        {
            Application.Current.Dispatcher.BeginInvoke((Action) (() =>
                                                                     {
                                                                         PropertyChangedEventHandler handler = PropertyChanged;
                                                                         if (handler != null) handler(this, new PropertyChangedEventArgs(propertyName));
                                                                     }));
        }
    }

Resizer Control:

<UserControl x:Class="MiscSamples.ResizerControl"
             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
             xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" 
             xmlns:d="http://schemas.microsoft.com/expression/blend/2008" 
             mc:Ignorable="d" 
             d:DesignHeight="300" d:DesignWidth="300">
    <Grid>
        <Thumb DragDelta="Center_DragDelta" Height="10" Width="10"
               VerticalAlignment="Center" HorizontalAlignment="Center"/>

        <Thumb DragDelta="UpperLeft_DragDelta" Height="10" Width="10"
               VerticalAlignment="Top" HorizontalAlignment="Left"/>

        <Thumb DragDelta="UpperRight_DragDelta" Height="10" Width="10"
               VerticalAlignment="Top" HorizontalAlignment="Right"/>

        <Thumb DragDelta="LowerLeft_DragDelta" Height="10" Width="10"
               VerticalAlignment="Bottom" HorizontalAlignment="Left"/>

        <Thumb DragDelta="LowerRight_DragDelta" Height="10" Width="10"
               VerticalAlignment="Bottom" HorizontalAlignment="Right"/>

    </Grid>
</UserControl>

Code Behind:

 public partial class ResizerControl : UserControl
    {
        public static readonly DependencyProperty XProperty = DependencyProperty.Register("X", typeof(double), typeof(ResizerControl), new FrameworkPropertyMetadata(0d,FrameworkPropertyMetadataOptions.BindsTwoWayByDefault));
        public static readonly DependencyProperty YProperty = DependencyProperty.Register("Y", typeof(double), typeof(ResizerControl), new FrameworkPropertyMetadata(0d, FrameworkPropertyMetadataOptions.BindsTwoWayByDefault));
        public static readonly DependencyProperty ItemWidthProperty = DependencyProperty.Register("ItemWidth", typeof(double), typeof(ResizerControl), new FrameworkPropertyMetadata(0d, FrameworkPropertyMetadataOptions.BindsTwoWayByDefault));
        public static readonly DependencyProperty ItemHeightProperty = DependencyProperty.Register("ItemHeight", typeof(double), typeof(ResizerControl), new FrameworkPropertyMetadata(0d, FrameworkPropertyMetadataOptions.BindsTwoWayByDefault));

        public double X
        {
            get { return (double) GetValue(XProperty); }
            set { SetValue(XProperty, value); }
        }

        public double Y
        {
            get { return (double)GetValue(YProperty); }
            set { SetValue(YProperty, value); }
        }

        public double ItemHeight
        {
            get { return (double) GetValue(ItemHeightProperty); }
            set { SetValue(ItemHeightProperty, value); }
        }

        public double ItemWidth
        {
            get { return (double) GetValue(ItemWidthProperty); }
            set { SetValue(ItemWidthProperty, value); }
        }

        public ResizerControl()
        {
            InitializeComponent();
        }

        private void UpperLeft_DragDelta(object sender, DragDeltaEventArgs e)
        {
            X = X + e.HorizontalChange;
            Y = Y + e.VerticalChange;

            ItemHeight = ItemHeight + e.VerticalChange * -1;
            ItemWidth = ItemWidth + e.HorizontalChange * -1;
        }

        private void UpperRight_DragDelta(object sender, DragDeltaEventArgs e)
        {
            Y = Y + e.VerticalChange;

            ItemHeight = ItemHeight + e.VerticalChange * -1;
            ItemWidth = ItemWidth + e.HorizontalChange;
        }

        private void LowerLeft_DragDelta(object sender, DragDeltaEventArgs e)
        {
            X = X + e.HorizontalChange;

            ItemHeight = ItemHeight + e.VerticalChange;
            ItemWidth = ItemWidth + e.HorizontalChange * -1;
        }

        private void LowerRight_DragDelta(object sender, DragDeltaEventArgs e)
        {
            ItemHeight = ItemHeight + e.VerticalChange;
            ItemWidth = ItemWidth + e.HorizontalChange;
        }

        private void Center_DragDelta(object sender, DragDeltaEventArgs e)
        {
            X = X + e.HorizontalChange;
            Y = Y + e.VerticalChange;
        }
    }