Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


Welcome, Guest
Guest Settings
Help

Thread: [Answered] Touch keyboard Windows 10



Permlink Replies: 12 - Last Post: Feb 21, 2018 5:51 PM Last Post By: Remy Lebeau (Te...
Primoz Cerar

Posts: 10
Registered: 6/4/06
[Answered] Touch keyboard Windows 10
Click to report abuse...   Click to reply to this thread Reply
  Posted: Dec 5, 2015 8:28 AM
Can someone explain to me why a delphi VCL (Metro or not) app does not open the system touch keyboard even when the setting in devices - typing to show the keyboard on desktop apps is turned on. Other desktop apps open it. Is this a delphi problem or a Windows Win32 API problem? I don't know if the apps that do open the keyboard use Win32 controls (Windows Edit Control) or something else.
Interestingly if you set a PasswordChar on a TEdit then the keyboard works as expected.

Edited by: Primoz Cerar on Apr 6, 2016 11:22 PM
James Silver

Posts: 7
Registered: 10/26/13
Re: Touch keyboard Windows 10
Click to report abuse...   Click to reply to this thread Reply
  Posted: Dec 17, 2015 12:24 PM   in response to: Primoz Cerar in response to: Primoz Cerar
I'm in the same situation as you. If you figure something out please post
here.
Primoz Cerar

Posts: 10
Registered: 6/4/06
Re: Touch keyboard Windows 10
Click to report abuse...   Click to reply to this thread Reply
  Posted: Dec 18, 2015 8:45 AM   in response to: Primoz Cerar in response to: Primoz Cerar
I still don't know the reason why this doesn't work. I was hoping for someone from Embarcadero to provide an explanation.

I did create a workaround for this, but it involves quite a bit of code which is currently not fully self contained. It involves a TScreen.OnActiveControlChange event passing changes to a thread that handles the verification of what needs to be done. Thread is needed because if the OnActiveControlChange event takes too long you loose mouse up/touch up events and get unpredictable behavior. I invoke the keyboard by executing TabTip.exe. Closing is done by posting a CM_CLOSE message to the keyboard window. There are checks done to see if physical keyboard is currently connected and tracking connect/disconnect of keyboard, whether device even has touch support, and finally processing of UI to prevent keyboard obscuring current input control is done on the root form class I use to create all other forms. I had to write some C API wrappers that are not available in Delphi to get physical keyboard connection info and touch keyboard events that provide keyboard size and location. The problems left are the fact that if the user closes the keyboard he has to change focus to another control before the keyboard will reappear and I did not figure out how to get the numeric keyboard automatically so the user has to click the numeric button on the keyboard when needed.

As you can see this is quite involved and if you decide to go down this road I can try to extract the necessary parts for you and post it here but I doubt it will be as simple as creating a class and adding a uses clause. It would require some more work for it to work like that. Also it currently only handles descendants of TCustomEdit (TEdit, TMemo, TDBEdit, TDBMemo, ... most DevExpress editors also descend from it) which was all that I needed, but you can easily add control classes you need.
James Silver

Posts: 7
Registered: 10/26/13
Re: Touch keyboard Windows 10
Click to report abuse...   Click to reply to this thread Reply
  Posted: Dec 18, 2015 9:14 AM   in response to: Primoz Cerar in response to: Primoz Cerar
Thanks for replying. In my opinion this is a Delphi or a windows 10 issue.
Other desktop application work just fine. I think I will reach over to
DevExpress and see if they can do some magic.
Primoz Cerar

Posts: 10
Registered: 6/4/06
Re: Touch keyboard Windows 10
Click to report abuse...   Click to reply to this thread Reply
  Posted: Dec 20, 2015 11:26 PM   in response to: James Silver in response to: James Silver
DevExpress builds their components on top of standard Delphi components. In other words if TEdit doesn't work then there is little to no chance that TcxEdit will work.
Primoz Cerar

Posts: 10
Registered: 6/4/06
Re: Touch keyboard Windows 10
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 26, 2016 12:27 AM   in response to: Primoz Cerar in response to: Primoz Cerar
I'm still waiting for someone from Embarcadero to provide some insight into this. As far as I understand it the edit control wrappers (TEdit, TMemo, ...) are missing some automation interface implementations that touch keyboard relies on.

Please confirm or correct me if I'm wrong and let me know if there are any plans to fix this. This is a big problem for touch applications as the provided Delphi touch keyboard control in my opinion is not the way to go when you have a system provided one (same as if you cold not get the system keyboard on Android or iOS).
David Millington

Posts: 257
Registered: 5/29/05
Re: Touch keyboard Windows 10
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 26, 2016 9:03 AM   in response to: Primoz Cerar in response to: Primoz Cerar
On 2016-02-26 08:27:53 +0000, Primoz Cerar said:

I'm still waiting for someone from Embarcadero to provide some insight
into this. As far as I understand it the edit control wrappers (TEdit,
TMemo, ...) are missing some automation interface implementations that
touch keyboard relies on.

Please confirm or correct me if I'm wrong and let me know if there are
any plans to fix this. This is a big problem for touch applications as
the provided Delphi touch keyboard control in my opinion is not the way
to go when you have a system provided one (same as if you cold not get
the system keyboard on Android or iOS).

Is there a QP report for this? That's step #1 to get them to change
anything. (http://quality.embarcadero.com.)
Primoz Cerar

Posts: 10
Registered: 6/4/06
Re: Touch keyboard Windows 10
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 28, 2016 2:24 AM   in response to: David Millington in response to: David Millington
I'm looking for an explanation first, so I know if there is a workaround before I post any change requests.
Tom Brunberg

Posts: 329
Registered: 12/27/04
Re: Touch keyboard Windows 10
Click to report abuse...   Click to reply to this thread Reply
  Posted: Mar 12, 2016 2:35 AM   in response to: Primoz Cerar in response to: Primoz Cerar
Primoz Cerar wrote:

I'm still waiting for someone from Embarcadero to provide some insight into this.

Please, remember though that these are peer to peer support forums. Therefore it is highly unlikely
that somebody from Embarcadero would see this post and respond.

To rattle the Emba cage you can
a) Open a product support case at http://www.embarcadero.com/support
b) File a bug report at https://quality.embarcadero.com/
c) Try to contact product manager Marco CantĂș (sorry, don't have a link)

--
Tom Brunberg
firstname.lastname@welho.com

Primoz Cerar

Posts: 10
Registered: 6/4/06
Re: Touch keyboard Windows 10
Click to report abuse...   Click to reply to this thread Reply
  Posted: Apr 6, 2016 11:11 PM   in response to: Primoz Cerar in response to: Primoz Cerar
It's time to answer my own question.

In order for Win32 controls to auto invoke the touch keyboard they need to implement IRawElementProviderSimple, ITextProvider and IValueProvider.
In short the control needs to return the IRawElementProviderSimple in response to a WM_GETOBJECT message and in IRawElementProviderSimple.GetPatternProvider it needs to return the ITextProvider or IValueProvider based on the patternId parameter.

This is required for proper UI Automation support that the touch keyboard depends on. I think this should have been already implemented in VCL if Embarcadero wants to claim that they support touch and Metro or Windows 10 style applications.
Achim Kalwa

Posts: 70
Registered: 10/22/99
Re: Touch keyboard Windows 10
Click to report abuse...   Click to reply to this thread Reply
  Posted: Apr 12, 2016 4:27 AM   in response to: Primoz Cerar in response to: Primoz Cerar
Hi Primoz,

Primoz Cerar wrote:

It's time to answer my own question.

In order for Win32 controls to auto invoke the touch keyboard they
need to implement IRawElementProviderSimple, ITextProvider and
IValueProvider. In short the control needs to return the
IRawElementProviderSimple in response to a WM_GETOBJECT message and
in IRawElementProviderSimple.GetPatternProvider it needs to return
the ITextProvider or IValueProvider based on the patternId
parameter.

This is required for proper UI Automation support that the touch
keyboard depends on. I think this should have been already
implemented in VCL if Embarcadero wants to claim that they support
touch and Metro or Windows 10 style applications.

This sounds very interesting. Can you show some example code for how to
implement that feature, i.e. a new control inherited from TEdit.

TIA
Achim

Primoz Cerar

Posts: 10
Registered: 6/4/06
Re: Touch keyboard Windows 10
Click to report abuse...   Click to reply to this thread Reply
  Posted: Apr 12, 2016 6:11 AM   in response to: Achim Kalwa in response to: Achim Kalwa
Not very simply unfortunately.
I started from this:
https://github.com/jhc-systems/DelphiUIAutomation
Which has the UIAutomation type library imported, then I updated TAutomatedEdit which already has IRawElementProviderSimple in IValueProvider implemented.
I added the ITextProvider implementation which requires an implementation of ITextRangeProvider (unit DelphiUIAutomation.TextRangeProviders).
WARNING! This is a quick and dirty (not even close to proper) implementation of the ITextProvider and ITextRangeProvider interfaces.
Also I may have missed something. Let me know if you can't get it to work.
If you want to set an input scope for a control (numeric, email, URL, ... keyboard) use the SetInputScope function in the Winapi.MsCTF unit. (This should be implemented as a property on the new control or in case of a VCL fix in the TCustomEdit class)

I don't know if this will fit in the post but here goes:

{***************************************************************************}
{ }
{ DelphiUIAutomation }
{ }
{ Copyright 2015 JHC Systems Limited }
{ }
{***************************************************************************}
{ }
{ Licensed under the Apache License, Version 2.0 (the "License"); }
{ you may not use this file except in compliance with the License. }
{ You may obtain a copy of the License at }
{ }
{ http://www.apache.org/licenses/LICENSE-2.0 }
{ }
{ Unless required by applicable law or agreed to in writing, software }
{ distributed under the License is distributed on an "AS IS" BASIS, }
{ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. }
{ See the License for the specific language governing permissions and }
{ limitations under the License. }
{ }
{***************************************************************************}
unit AutomatedEdit;

interface

uses
UIAutomationCore_TLB,
messages,
ActiveX,
System.SysUtils, System.Classes, Vcl.Controls, Vcl.StdCtrls,
DelphiUIAutomation.TextRangeProviders;

type
TAutomatedEdit = class(TEdit,
ITextProvider,
IValueProvider,
IRawElementProviderSimple)
private
{ Private declarations }
FRawElementProviderSimple : IRawElementProviderSimple;

procedure WMGetObject(var Message: TMessage); message WM_GETOBJECT;

protected
{ Protected declarations }
public
{ Public declarations }

// IRawElementProviderSimple
function Get_ProviderOptions(out pRetVal: ProviderOptions): HResult; stdcall;
function GetPatternProvider(patternId: SYSINT; out pRetVal: IUnknown): HResult; stdcall;
function GetPropertyValue(propertyId: SYSINT; out pRetVal: OleVariant): HResult; stdcall;
function Get_HostRawElementProvider(out pRetVal: IRawElementProviderSimple): HResult; stdcall;

// IValueProvider
function SetValue(val: PWideChar): HResult; stdcall;
function Get_Value(out pRetVal: WideString): HResult; stdcall;
function Get_IsReadOnly(out pRetVal: Integer): HResult; stdcall;

// ITextProvider
function GetSelection(out pRetVal: PSafeArray): HResult; stdcall;
function GetVisibleRanges(out pRetVal: PSafeArray): HResult; stdcall;
function RangeFromChild(const childElement: IRawElementProviderSimple;
out pRetVal: ITextRangeProvider): HResult; stdcall;
function RangeFromPoint(point: UiaPoint; out pRetVal: ITextRangeProvider): HResult; stdcall;
function Get_DocumentRange(out pRetVal: ITextRangeProvider): HResult; stdcall;
function Get_SupportedTextSelection(out pRetVal: SupportedTextSelection): HResult; stdcall;
published
{ Published declarations }
end;

procedure Register;

implementation

uses
windows;

procedure Register;
begin
RegisterComponents('Samples', [TAutomatedEdit]);
end;

{ TAutomatedEdit }

function TAutomatedEdit.GetPatternProvider(patternId: SYSINT;
out pRetVal: IInterface): HResult;
begin
result := S_OK;
pRetval := nil;

if (patternID = UIA_ValuePatternID) then
begin
pRetVal := IValueProvider(self);
end else if (patternID = UIA_TextPatternID) then
begin
pRetVal := ITextProvider(self);
end
end;

function TAutomatedEdit.GetPropertyValue(propertyId: SYSINT;
out pRetVal: OleVariant): HResult;
begin
if(propertyId = UIA_ClassNamePropertyId) then
begin
TVarData(pRetVal).VType := varOleStr;
TVarData(pRetVal).VOleStr := pWideChar(self.ClassName);
result := S_OK;
end
else if(propertyId = UIA_NamePropertyId) then
begin
TVarData(pRetVal).VType := varOleStr;
TVarData(pRetVal).VOleStr := pWideChar(self.Name);
result := S_OK;
end
else if(propertyId = UIA_AutomationIdPropertyId) then
begin
TVarData(pRetVal).VType := varOleStr;
TVarData(pRetVal).VOleStr := pWideChar(self.Name);
result := S_OK;
end
else if(propertyId = UIA_ControlTypePropertyId) then
begin
TVarData(pRetVal).VType := varInteger;
TVarData(pRetVal).VInteger := UIA_EditControlTypeId;
result := S_OK;
end
else
result := S_FALSE;
end;

function TAutomatedEdit.GetSelection(out pRetVal: PSafeArray): HResult;
var
I: Integer;
ptr: Pointer;
rp: TCustomEditRangeProvider;
begin
if (SelStart = 1) and (SelLength = Length(Text)) then begin
pRetVal := SafeArrayCreateVector(VT_PTR, 0, 1);
I := 0;
rp := TCustomEditRangeProvider.Create(Self);
ptr := @rp;
SafeArrayPutElement(pRetVal, I, ptr);
end else
pRetVal := SafeArrayCreateVector(VT_PTR, 0, 0);
Result := S_OK;
end;

function TAutomatedEdit.GetVisibleRanges(out pRetVal: PSafeArray): HResult;
var
I: Integer;
ptr: Pointer;
rp: TCustomEditRangeProvider;
begin
pRetVal := SafeArrayCreateVector(VT_PTR, 0, 1);
I := 0;
rp := TCustomEditRangeProvider.Create(Self);
ptr := @rp;
SafeArrayPutElement(pRetVal, I, ptr);
Result := S_OK;
end;

function TAutomatedEdit.Get_DocumentRange(
out pRetVal: ITextRangeProvider): HResult;
begin
pRetVal := ITextRangeProvider(TCustomEditRangeProvider.Create(Self));
Result := S_OK;
end;

function TAutomatedEdit.Get_HostRawElementProvider(
out pRetVal: IRawElementProviderSimple): HResult;
begin
result := UiaHostProviderFromHwnd (self.Handle, pRetVal);
end;

function TAutomatedEdit.Get_IsReadOnly(out pRetVal: Integer): HResult;
begin
pRetVal := 0; // Maybe?
Result := S_OK;
end;

function TAutomatedEdit.Get_ProviderOptions(
out pRetVal: ProviderOptions): HResult;
begin
pRetVal:= ProviderOptions_ServerSideProvider;
Result := S_OK;
end;

function TAutomatedEdit.Get_SupportedTextSelection(
out pRetVal: SupportedTextSelection): HResult;
begin
pRetVal := SupportedTextSelection_Single;
Result := S_OK;
end;

function TAutomatedEdit.Get_Value(out pRetVal: WideString): HResult;
begin
Result := S_OK;
pRetVal := self.Text;
end;

function TAutomatedEdit.RangeFromChild(
const childElement: IRawElementProviderSimple;
out pRetVal: ITextRangeProvider): HResult;
begin
pRetVal := ITextRangeProvider(TCustomEditRangeProvider.Create(Self));
Result := S_OK;
end;

function TAutomatedEdit.RangeFromPoint(point: UiaPoint;
out pRetVal: ITextRangeProvider): HResult;
begin
pRetVal := ITextRangeProvider(TCustomEditRangeProvider.Create(Self));
Result := S_OK;
end;

function TAutomatedEdit.SetValue(val: PWideChar): HResult;
begin
self.Text := val;
Result := S_OK;
end;

procedure TAutomatedEdit.WMGetObject(var Message: TMessage);
begin
if (Message.Msg = WM_GETOBJECT) then
begin
QueryInterface(IID_IRawElementProviderSimple, FRawElementProviderSimple);

message.Result := UiaReturnRawElementProvider(self.Handle, Message.WParam, Message.LParam, FRawElementProviderSimple);
end
else
Message.Result := DefWindowProc(self.Handle, Message.Msg, Message.WParam, Message.LParam);
end;

end.

**************************************************************
The ITextRangeProvider implementation:
unit DelphiUIAutomation.TextRangeProviders;

interface

uses
UIAutomationCore_TLB, ActiveX, StdCtrls, Windows, SysUtils;

type
TCustomEditRangeProvider = class(TInterfacedObject, ITextRangeProvider)
private
FCustomEdit: TCustomEdit;
protected
function GetCustomEdit: TCustomEdit;
public
constructor Create(CustomEdit: TCustomEdit);
function Clone(out pRetVal: ITextRangeProvider): HResult; stdcall;
function Compare(const range: ITextRangeProvider; out pRetVal: Integer): HResult; stdcall;
function CompareEndpoints(endpoint: TextPatternRangeEndpoint;
const targetRange: ITextRangeProvider;
targetEndpoint: TextPatternRangeEndpoint; out pRetVal: SYSINT): HResult; stdcall;
function ExpandToEnclosingUnit(unit_: TextUnit): HResult; stdcall;
function FindAttribute(attributeId: SYSINT; val: OleVariant; backward: Integer;
out pRetVal: ITextRangeProvider): HResult; stdcall;
function FindText(const text: WideString; backward: Integer; ignoreCase: Integer;
out pRetVal: ITextRangeProvider): HResult; stdcall;
function GetAttributeValue(attributeId: SYSINT; out pRetVal: OleVariant): HResult; stdcall;
function GetBoundingRectangles(out pRetVal: PSafeArray): HResult; stdcall;
function GetEnclosingElement(out pRetVal: IRawElementProviderSimple): HResult; stdcall;
function GetText(maxLength: SYSINT; out pRetVal: WideString): HResult; stdcall;
function Move(unit_: TextUnit; count: SYSINT; out pRetVal: SYSINT): HResult; stdcall;
function MoveEndpointByUnit(endpoint: TextPatternRangeEndpoint; unit_: TextUnit; count: SYSINT;
out pRetVal: SYSINT): HResult; stdcall;
function MoveEndpointByRange(endpoint: TextPatternRangeEndpoint;
const targetRange: ITextRangeProvider;
targetEndpoint: TextPatternRangeEndpoint): HResult; stdcall;
function Select: HResult; stdcall;
function AddToSelection: HResult; stdcall;
function RemoveFromSelection: HResult; stdcall;
function ScrollIntoView(alignToTop: Integer): HResult; stdcall;
function GetChildren(out pRetVal: PSafeArray): HResult; stdcall;
property CustomEdit: TCustomEdit read GetCustomEdit;
end;

implementation

{ TCustomEditRangeProvider }

function TCustomEditRangeProvider.AddToSelection: HResult;
begin
FCustomEdit.SelectAll;
Result := S_OK;
end;

function TCustomEditRangeProvider.Clone(
out pRetVal: ITextRangeProvider): HResult;
begin
pRetVal := ITextRangeProvider(TCustomEditRangeProvider.Create(FCustomEdit));
Result := S_OK;
end;

function TCustomEditRangeProvider.Compare(const range: ITextRangeProvider;
out pRetVal: Integer): HResult;
begin
pRetVal := 1;
Result := S_OK;
end;

function TCustomEditRangeProvider.CompareEndpoints(
endpoint: TextPatternRangeEndpoint; const targetRange: ITextRangeProvider;
targetEndpoint: TextPatternRangeEndpoint; out pRetVal: SYSINT): HResult;
begin
pRetVal := 0;
Result := S_OK;
end;

constructor TCustomEditRangeProvider.Create(CustomEdit: TCustomEdit);
begin
FCustomEdit := CustomEdit;
end;

function TCustomEditRangeProvider.ExpandToEnclosingUnit(
unit_: TextUnit): HResult;
begin
Result := S_OK;
end;

function TCustomEditRangeProvider.FindAttribute(attributeId: SYSINT;
val: OleVariant; backward: Integer; out pRetVal: ITextRangeProvider): HResult;
begin
pRetVal := ITextRangeProvider(Self);
Result := S_OK
end;

function TCustomEditRangeProvider.FindText(const text: WideString; backward,
ignoreCase: Integer; out pRetVal: ITextRangeProvider): HResult;
begin
pRetVal := ITextRangeProvider(Self);
Result := S_OK
end;

function TCustomEditRangeProvider.GetAttributeValue(attributeId: SYSINT;
out pRetVal: OleVariant): HResult;
begin
pRetVal := ITextRangeProvider(Self);
Result := S_OK
end;

function TCustomEditRangeProvider.GetBoundingRectangles(
out pRetVal: PSafeArray): HResult;
begin
pRetVal := SafeArrayCreateVector(VT_RECORD, 0, 0);
Result := S_OK;
end;

function TCustomEditRangeProvider.GetChildren(out pRetVal: PSafeArray): HResult;
begin
pRetVal := SafeArrayCreateVector(VT_PTR, 0, 0);
Result := S_OK;
end;

function TCustomEditRangeProvider.GetCustomEdit: TCustomEdit;
begin
Result := FCustomEdit;
end;

function TCustomEditRangeProvider.GetEnclosingElement(
out pRetVal: IRawElementProviderSimple): HResult;
begin
if not Supports(FCustomEdit, IID_IRawElementProviderSimple, pRetVal) then
Result := S_OK
else
Result := E_NOINTERFACE;
end;

function TCustomEditRangeProvider.GetText(maxLength: SYSINT;
out pRetVal: WideString): HResult;
begin
if maxLength < 0 then
pRetVal := FCustomEdit.Text
else
pRetVal := Copy(FCustomEdit.Text, 1, maxLength);
Result := S_OK;
end;

function TCustomEditRangeProvider.Move(unit_: TextUnit; count: SYSINT;
out pRetVal: SYSINT): HResult;
begin
pRetVal := 0;
Result := S_OK;
end;

function TCustomEditRangeProvider.MoveEndpointByRange(
endpoint: TextPatternRangeEndpoint; const targetRange: ITextRangeProvider;
targetEndpoint: TextPatternRangeEndpoint): HResult;
begin
Result := S_OK;
end;

function TCustomEditRangeProvider.MoveEndpointByUnit(
endpoint: TextPatternRangeEndpoint; unit_: TextUnit; count: SYSINT;
out pRetVal: SYSINT): HResult;
begin
pRetVal := 0;
Result := S_OK;
end;

function TCustomEditRangeProvider.RemoveFromSelection: HResult;
begin
Result := S_OK
end;

function TCustomEditRangeProvider.ScrollIntoView(alignToTop: Integer): HResult;
begin
Result := S_OK;
end;

function TCustomEditRangeProvider.Select: HResult;
begin
FCustomEdit.SelectAll;
Result := S_OK;
end;

end.
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Touch keyboard Windows 10
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 21, 2018 5:51 PM   in response to: Primoz Cerar in response to: Primoz Cerar
Primoz Cerar wrote:
In order for Win32 controls to auto invoke the touch keyboard they need to implement IRawElementProviderSimple, ITextProvider and IValueProvider.
In short the control needs to return the IRawElementProviderSimple in response to a WM_GETOBJECT message and in IRawElementProviderSimple.GetPatternProvider it needs to return the ITextProvider or IValueProvider based on the patternId parameter.

This is required for proper UI Automation support that the touch keyboard depends on. I think this should have been already implemented in VCL if Embarcadero wants to claim that they support touch and Metro or Windows 10 style applications.

Sounds like something that Microsoft should have implemented in the underlying EDIT control, then individual apps (ie, the VCL) wouldn't have to implement it manually at all.

--
Remy Lebeau (TeamB)
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02