WWDC Quick Look 💓 By SwiftGGTeam
中文 EN 日本語
Design for the iPadOS pointer

Design for the iPadOS pointer

WWDC 2020DesigniPadOSPointer
Watch original video

Highlight

The iPadOS pointer dynamically adjusts precision across buttons, text, custom areas, and gestures with Adaptive Precision, and adapts the touch interface for indirect input with Highlight, Lift, Hover, custom shape, and trackpad gestures.

Core Content

The iPad’s interface is built for your fingers first. Buttons have larger click areas, space is left between controls, and dragging and selecting both accept a fingertip-sized input by default. When the trackpad comes to iPadOS, the question isn’t just “can it click?” Traditional desktop pointers are accurate to pixels, but many iPad controls only need to know which button, which line of text, or which time period the user clicked.

Apple’s answer is Adaptive Precision. The pointer is usually a circle close to the size of a fingertip; when it is close to a button, it will snap to the button and change into the shape of the button; when moved to the text, it will turn into a beam that matches the line height; when entering the custom schedule view, it can snap to 15-minute blocks. The accuracy of the pointer changes with the semantics of the interface, and the user does not have to stop the mouse on the boundary of a few pixels.

This session breaks down the pointers into three layers. The first level is the design principle: the pointer should complement touch, not replace it. The second layer is visual effects: Highlight, Lift, Hover and custom shape respectively serve controls of different sizes, backgrounds and interaction densities. The third layer is interaction acceleration: hover can expose the toolbar in advance, secondary click can directly open the context menu, and two-finger gestures must take effect relative to the view where the pointer is located.

If your app already makes heavy use of standard UIKit controls, basic pointer behavior for buttons and text is available for free. The areas that really require design judgment are customized canvases, dense toolbars, table borders, maps, readers, and schedule interfaces. These areas require you to decide where the pointer should snap to, what shape it should take, whether to show hover feedback, and how trackpad and touch gestures should be consistent.

Detailed Content

1. Adaptive Precision solves the problem of delay first

(03:36) There’s a precision mismatch between traditional pointers and iPad controls: the pointer moves by pixels, and buttons only care about which area they hit. iPadOS designs the pointer to be circular by default, and when it is close to the button group, it will be attracted to the button and transform into the button outline. The shape change itself is feedback, telling the user which control is currently targeted.

(05:10) Text selection uses another precision. After the pointer enters the TextView, it becomes a beam that matches the row height. It is adsorbed by row in the vertical direction and retains pixel-level accuracy in the horizontal direction. This will not only prevent you from clicking on the previous or next line, but also place the insertion point at a specific character position.

(06:55) Standard UIKit controls have direct access to button and text behaviors. Custom views need to express their own semantics. The session schedule view example snaps vertically to a 15-minute range, allowing users to know in advance whether the start time will fall at 2:15 or 2:30 when dragging and creating an event.

2. Behind the pointer are model pointer, snapping, centering and magnetism

(08:50) What you see on the screen is a visible pointer, and there is an invisible model pointer inside the system. The model pointer records the real position and is used to determine which hit area the hover hit. The visible pointer can animate to the target control, while using parallax to indicate that the real position is still on the edge.

(10:11) Recentering will center the pointer on the control after the user lifts their finger, reducing the probability of leaving the button due to slight scrolling of the finger on the next click. The pointer will also be automatically hidden to avoid constantly reminding users to “use the trackpad now”, allowing natural switching between touch, Apple Pencil and pointer.

(12:10) iPadOS reuses the acceleration curve of macOS and adds inertia and magnetism. Inertia allows users to span greater distances with one quick swipe; magnetism scans nearby targets based on projected position and sliding direction, and places the pointer on the most likely snappable element. Any control that supports pointer snapping will get this behavior.

3. Three pointer effects correspond to different control sizes

(14:23) Pointer Effect is the synthesis of a focus state between the pointer and the content when hovering. Highlight is suitable for small controls without background, such as bar button and tab bar; the pointer will move behind the content, become the background of the control, and connect the gesture position through parallax and light effect.

(15:40) Lift is suitable for medium-sized elements that already have backgrounds, such as App icons and Control Center modules. The pointer disappears behind the element, the element enlarges, casts a shadow, and the color creates radiosity on the background. The specular light also performs a second task: showing the true position of the underlying pointer.

(17:16) Hover is suitable for larger objects that would perform poorly if they had pointers directly changing to their shape. Hover changes only the appearance of the object, leaving the pointer in its default shape and staying over the object. It can also combine scale, shadow, tint and custom shapes.

4. Try Automatic Effect first, then deal with consistency and hit region

(17:55) Apple recommends using Automatic Effect first. Effects are selected based on object type, location, size, and shape, and these rules will be updated in the future. Manual specification is only needed if the automatic result is inconsistent with the context.

(18:31) The bookmark icon of Books looks like it can use Lift, because it has its own background; but it is located in the toolbar, and the buttons next to it use Highlight. To be consistent with the group, it should also use Highlight. The selected option of segmented control is a counterexample: it would be redundant to wrap Highlight around a rectangular object, and Lift would be more appropriate.

(20:01) The hit region affects the activation range of the pointer effect. Too small will make the user feel they have to be very precise, too large will make it difficult for the pointer to disengage. Elements with bezels can consider about 12 points of padding, and elements without bezels can consider about 24 points of padding. Do not leave holes between objects in the same group to prevent the pointer from returning to a circle in the middle and causing unnecessary animation.

5. Custom pointer allows the shape to directly express the intention

(24:28) The iPadOS pointer’s material automatically adjusts color based on the background, maintaining contrast while not blocking content. When it changes into a button shape, the color will brighten to ensure the label is readable; when clicked, the circle will shrink and darken to give clear click feedback.

(25:40) The default circle represents touch. It can become a rounded rectangle, a vertical bar, a horizontal pill, or any shape. A cross shape can represent a higher-precision area; a pair of triangles can represent the drag direction; and when you hold down the Command key to change the pointer function, the shape can also indicate mode changes.

(28:03) The shape of the custom pointer should be simple and easy to understand, and try to use solid shape. The default pointer is 19 point circle; narrow shapes need to be resized to maintain visual weight, and the cross mark in the session is 24 points. Be careful with the anchor point: directional shapes will make the deformation appear eccentric. When it can be expressed with a centered target shape, give priority to maintaining the central anchor point.

(30:02) The example of reading and annotating text is very specific. Instead of making the pointer a marker pen glyph, it’s better to make it a marker tip shape. The latter also expresses the text selection height and is consistent with the snapping and adaptive precision behaviors of I-beam. Apple also cautions against creating custom pointers for decorations; users will interpret pointer changes as available functionality.

6. Pointer interaction can speed up existing operations

(32:50) Pointers bring greater precision, but they should not force apps to redo their layout, increase density, or add functionality that can only be accomplished with pointers. Numbers’ column width adjustment example illustrates the correct direction: when using a finger, the user first selects the column header and then drags the handle; when using a pointer, the user can move directly to the dividing line between columns and drag, which has the same function but a shorter path.

(36:24) pointer hover is a signal of intent. Books can hide the toolbar when reading and display it immediately when the pointer moves to the upper area; full-screen playback or immersive experience can also restore the control bar when pointer movement appears. Click and drag can also directly enter drag-and-drop or drag select, eliminating the long press and wait required to distinguish scrolling in touch scenes.

(37:50) One-finger interactions are mainly left to pointing and clicking, three-finger gestures are left to the system, and most apps should focus on the two-finger trackpad gesture. The gesture should occur on the view below the pointer. The Maps example shows that two-finger scrolling, zooming, and rotating are all anchored to the pointer position, like placing two fingers at the same location on the screen.

(39:35) A secondary click can come from a trackpad two-finger click, a secondary mouse button, or a Control-click. By default, the system uses it to display the context menu immediately. For custom interfaces, it’s suitable for accelerating alternative operations that would otherwise require a long press or special mode to trigger.

Core Takeaways

  • Interval snapping for schedules and timelines: What: Make the pointer when creating events, editing clips, or dragging time blocks snap to an actually acceptable time granularity. Why it’s worth doing: The session’s schedule view aligns the start time to a 15-minute interval, reducing the user’s guesswork when dragging to the boundary. How ​​to start: First define the range accepted by the business, and then make the hover feedback and drag preview always fall on the same set of ranges.

  • Make readers and players expose control bars with hover: What: The content area remains immersive and the controls are revealed when the pointer moves to the toolbar area or starts moving. Why it’s worth doing: Books example shows that pointer position can express the next step intention, and the user does not need to click once to retrieve the tool. How ​​to get started: Adjust the control bar display rule from tap-only to a combination of hover region, pointer movement and inactivity timer.

  • Design marker tip pointer for text markup tool**: What to do: In the reading, annotation, PDF or code review tool, make the highlight tool pointer into a tip shape that can represent the height of the selection. Why it’s worth doing: The session explicitly favors marker tip as a superior solution to marker pen glyph because it expresses both selection height and I-beam-like behavior. How ​​to start: Use a solid shape close to the default pointer visual weight, keep the center anchor, and make it appear only in areas that really change the selection behavior.

  • Expose tables and canvases to precise drag entries: What: Column width dividers, cropping boundaries, control points and small handles are directly draggable when the pointer is close. Why it’s worth doing: The Numbers example does not add new features, it just uses pointer precision to skip the “select first and then find the handle” step. How ​​to start: Find the drag operations in the touch path that require a long press, select first or open mode, and add hover feedback and a more appropriate adsorption range to these boundaries.

  • Bind the two-finger gesture to the object under the pointer: What to do: In maps, whiteboards, image editors, or multi-column interfaces, two-finger scrolling, zooming, and rotation all act on the view where the pointer is located. Why it’s worth doing: The Maps example shows that after the pointer is used as a gesture anchor, the user can naturally switch the operation object between the map and the card. How ​​to start: First clarify which two-finger gestures are supported in each area, and then ensure that scaling and rotation occur around the pointer location.

Comments

GitHub Issues · utterances