Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


Welcome, Guest
Guest Settings
Help

Thread: Writing a Telnet Relay Application



Permlink Replies: 22 - Last Post: Aug 17, 2017 9:11 PM Last Post By: Remy Lebeau (Te...
Adam Hair

Posts: 18
Registered: 7/26/17
Writing a Telnet Relay Application
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 9, 2017 6:07 PM
Hi,

I have a situation where I wish to connect to a device using Telnet to read data. The device only allows a single connection, however we are wanting multiple users to be able to read simultaneously. I'm therefore looking at creating a relay application that will connect to the server using TidTelnet, and then relay the messages to any connections using TidTelnetServer.

However I am having some problems trying to get this to work. (I'm not very cluey when it comes to indy components or winsock at all for that matter).

I can establish the connection with the TidTelnet client using the TidTelnet.DataAvailable event.

However I'm not sure how to relay this through to the tIdTelnetServer component.

This is where I've gotten to so far - but I know that this is wrong. (Because it's not working).

procedure TMainForm.IdTelnet1DataAvailable(Sender: TIdTelnet; const Buffer: TIdBytes);
var
i : integer;
AContext: TIdContext;
begin
for I := 0 to IdTelnetServer1.Contexts.count - 1 do
begin
AContext := TidContext(IDTelnetServer1.Contexts.LockList.items[i]);
AContext.Connection.IOHandler.Write(Buffer);
end;
end;

Can someone please point me in the right direction?

(This only needs to have unidirectional communication - there's no need for clients to be able to transmit back to the original device).

Thanks & Regards

Adam.
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Writing a Telnet Relay Application
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 10, 2017 12:01 PM   in response to: Adam Hair in response to: Adam Hair
Adam Hair wrote:

I can establish the connection with the TidTelnet client using the
TidTelnet.DataAvailable event.

However I'm not sure how to relay this through to the tIdTelnetServer
component.

I have posted examples many many times before showing how to safely
broadcast data to multiple clients connected to Indy TCP server
components.

This is where I've gotten to so far - but I know that this is wrong.
(Because it's not working).

You are not accessing the server's Contexts list correctly. You are
locking it on each loop iteration, but not unlocking it. The code
needs to look more like this instead:

procedure TMainForm.IdTelnet1DataAvailable(Sender: TIdTelnet; const
Buffer: TIdBytes);
var
  list: TIdContextList;
  i : integer;
  AContext: TIdContext;
begin
  list := IdTelnetServer1.Contexts.LockList;
  try
    for I := 0 to list.count - 1 do
    begin
      AContext := TIdContext(list.items[i]);
      try
        AContext.Connection.IOHandler.Write(Buffer);
      except
      end;
    end;
  finally
    IdTelnetServer1.Contexts.UnlockList;
  end;
end;


This approach will work only if the server does not send any other data
to the clients. If it does (for instance, TIdTelnetServer does perform
login negotiation before the OnExecute event is fired), this approach
is a race condition that can corrupt communications if relay data is
sent at the same time as other data.

A safer option is to use the TIdContext.Data property (or better,
derive a class from TIdTelnetServerContext) to hold a thread-safe queue
of outbound data. Your TIdTelnet.OnDataAvailable event can push relay
data into each client's queue, and then you can have the
TIdTelnetServer.OnExecute event send the data to each client when it is
safe to do so. For example:

uses
  ...,
  IdThreadSafe, SyncObjs;
 
type
  TMyContext = class(TIdTelnetServerContext)
  public
    Queue: TIdThreadSafeList;
    QueueHasData: TEvent;
    constructor Create(AConnection: TIdTCPConnection; AYarn: TIdYarn;
AList: TIdContextThreadList = nil); override;
    destructor Destroy; override;
  end;
 
...
 
type
  PIdBytes := ^TIdBytes;
 
constructor TMyContext.Create(AConnection: TIdTCPConnection; AYarn:
TIdYarn; AList: TIdContextThreadList = nil);
begin
  inherited;
  Queue := TIdThreadSafeList.Create;
  QueueHasData := TEvent.Create(nil, True, False, '');
end;
 
destructor TMyContext.Destroy;
var
  list: TList;
  i: Integer;
begin
  list := Queue.LockList;
  try
    while list.Count > 0 do
    begin
      Dispose(PIdBytes(list[0]));
      list.Delete(0);
    end;
  finally
    Queue.UnlockList;
  end;
  Queue.Free;
  QueueHasData.Free;
  inherited;
end;
 
procedure TMainForm.FormCreate(Sender: TObject);
begin
  // set this before acticating the server...
  IdTelnetServer1.ContextClass := TMyContext;
end;
 
procedure TMainForm.IdTelnet1DataAvailable(Sender: TIdTelnet; const
Buffer: TIdBytes);
var
  CtxList: TIdContextList;
  Ctx: TMyContext;
  QueueList: TList;
  Data: PIdBytes;
  i : integer;
begin
  CtxList := IdTelnetServer1.Contexts.LockList;
  try
    for i := 0 to CtxList.count-1 do
    begin
      Ctx := TMyContext(TIdContext(CtxList[i]));
      try
        New(Data);
        Data^ := Buffer; // TIdBytes is reference-counted...
        QueueList := Ctx.Queue.LockList;
        try
          QueueList.Add(Data);
          Ctx.QueueHasData.SetEvent;
        finally
          Ctx.Queue.UnlockList;
        end;
      except
      end;
    end;
  finally
    IdTelnetServer1.Contexts.UnlockList;
  end;
end;
 
procedure TMainForm.IdTelnetServer1Execute(AContext: TIdContext);
var
  Ctx: TMyContext;
  list: TList;
  Data: PIdBytes;
  i : integer;
begin
  Ctx := TMyContext(AContext);
 
  if Ctx.QueueHasData.WaitFor(0) = wrSignaled then
  begin
    list := Ctx.Queue.LockList;
    try
      while list.Count > 0 do
      begin
        Data := PIdBytes(list[i]);
        list.Delete(0);
        try
          AContext.Connection.IOHandler.Write(Data^);
        finally
          Dispose(Data);
        end;
      end;
    finally
      if list.Count = 0 then
        Ctx.QueueHasData.ResetEvent;
      Ctx.Queue.UnlockList;
    end;
  end;
 
  if AContext.Connection.IOHandler.InputBufferIsEmpty then
  begin
    AContext.Connection.IOHandler.CheckForDataOnSource(0);
    AContext.Connection.IOHandler.CheckForDisconnect;
  end;
 
  if not AContext.Connection.IOHandler.InputBufferIsEmpty then
  begin
    // handle inbound client data/command as needed...
    AContext.Connection.IOHandler.InputBuffer.Clear;
  end;
end;


--
Remy Lebeau (TeamB)
Adam Hair

Posts: 18
Registered: 7/26/17
Re: Writing a Telnet Relay Application
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 10, 2017 4:05 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Hi Remy,

Thank you for your detailed reply!

I ended up going with the example you gave,using a class from TIdTelnetServerContext as I want to try and learn the best approach. :-)

I had to change:
while list.Count > 0 do
to:
for i := list.count - 1 downto 0 do

(I was getting index out of bounds errors as the variable i was not initialised in the IdTelnetServer1Execute event and assumed this is what was supposed to be in there. (Please let me know if this is incorrect).

This appears to be working now - however after an extended run of the application I notice that I get invalid pointer operation error messages which then causes the app to crash. I'm able to escellate the error / cause it to happen more quickly if I increase the data that is being transmitted by the original device.

Just wondering if you have any ideas what might be causing this?

Thanks again, and have a great weekend!

Adam.

Edited by: Adam Hair on Aug 10, 2017 4:16 PM

Edited by: Adam Hair on Aug 10, 2017 5:51 PM
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Writing a Telnet Relay Application [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 11, 2017 1:03 PM   in response to: Adam Hair in response to: Adam Hair
Adam Hair wrote:

I had to change:
while list.Count > 0 do
to:
for i := list.count - 1 downto 0 do

(I was getting index out of bounds errors as the variable i was not
initialised in the IdTelnetServer1Execute event

That was my bad, sorry. The line was supposed to be accessing list[0]
instead of list[i]. Remove 'i' altogether.

assumed this is what was supposed to be in there. (Please let me know
if this is incorrect)

It is incorrect. By changing the code to use a backwards 'for' loop,
you may have fixed the bounds error, but you have introduced a new
logic bug, as you are now sending data to the client in the wrong
order. The bytes need to be sent in the order they were queued, not in
the reverse order. If you are going to use a 'for' loop, use a
forwards loop instead:

for i := 0 to list.count - 1 do


I chose to use a 'while' loop instead of a 'for' loop so that items
could be removed from the queue as they are being sent, and unsent
items are left in the queue (in case an exception is raised). If you
use a 'for' loop, you need a keep track of how many items are
successfully sent and then run a second loop to remove those items,
which I felt was overkill.

This appears to be working now - however after an extended run of the
application I notice that I get invalid pointer operation error
messages which then causes the app to crash. I'm able to escellate
the error / cause it to happen more quickly if I increase the data
that is being transmitted by the original device.

Just wondering if you have any ideas what might be causing this?

Not without knowing where the exception is actually occuring. And it
should't be crashing the entire app, since most of the code is isolated
to individual threads. Exceptions can't spawn across thread
boundaries. But they can corrupt memory that affects other threads.
So, just about anything can happen, I suppose. You are just going to
have to debug the code.

I am doing some tricky things with pointers, sincee you didn't indicate
which version of Delphi you are using, so I wanted the code to be
usable in a wide range of versions. If you are using a version of
Delphi that supports Generics, you can make the code a little safer by
using TList<T> or TQueue<T> instead of TIdThreadSafeList, eg:

uses
  ...,
  System.Generics.Collections, System.SyncObjs;
 
type
  TMyContext = class(TIdTelnetServerContext)
  public
    Queue: TQueue<TIdBytes>;
    QueueLock: TCriticalSection;
    QueueHasData: TEvent;
    constructor Create(AConnection: TIdTCPConnection; AYarn: TIdYarn; 
AList: TIdContextThreadList = nil); override;
    destructor Destroy; override;
  end;
 
...
 
constructor TMyContext.Create(AConnection: TIdTCPConnection; AYarn:
TIdYarn; AList: TIdContextThreadList = nil);
begin
  inherited;
  Queue := TQueue<TIdBytes>.Create;
  QueueLock := TCriticalSection.Create;
  QueueHasData := TEvent.Create(nil, True, False, '');
end;
 
destructor TMyContext.Destroy;
begin
  Queue.Free;
  QueueLock.Free;
  QueueHasData.Free;
  inherited;
end;
 
procedure TMainForm.FormCreate(Sender: TObject);
begin
  // set this before acticating the server...
  IdTelnetServer1.ContextClass := TMyContext;
end;
 
procedure TMainForm.IdTelnet1DataAvailable(Sender: TIdTelnet; const
Buffer: TIdBytes);
var
  list: TIdContextList;
  Ctx: TMyContext;
  i : integer;
begin
  list := IdTelnetServer1.Contexts.LockList;
  try
    for i := 0 to list.Count-1 do
    begin
      Ctx := TMyContext(TIdContext(list[i]));
      try
        Ctx.QueueLock.Enter;
        try
          Ctx.Queue.Enqueue(Buffer);
          Ctx.QueueHasData.SetEvent;
        finally
          Ctx.QueueLock.Leave;
        end;
      except
      end;
    end;
  finally
    IdTelnetServer1.Contexts.UnlockList;
  end;
end;
 
procedure TMainForm.IdTelnetServer1Execute(AContext: TIdContext);
var
  Ctx: TMyContext;
  i : integer;
begin
  Ctx := TMyContext(AContext);
 
  if Ctx.QueueHasData.WaitFor(0) = wrSignaled then
  begin
    Ctx.QueueLock.Enter;
    try
      while Ctx.Queue.Count > 0 do
        AContext.Connection.IOHandler.Write(Ctx.Queue.Dequeue);
    finally
      if Ctx.Queue.Count = 0 then
        Ctx.QueueHasData.ResetEvent;
      Ctx.QueueLock.Leave;
    end;
  end;
 
  if AContext.Connection.IOHandler.InputBufferIsEmpty then
  begin
    AContext.Connection.IOHandler.CheckForDataOnSource(0);
    AContext.Connection.IOHandler.CheckForDisconnect;
  end;
 
  if not AContext.Connection.IOHandler.InputBufferIsEmpty then
  begin
    // handle inbound client data/command as needed...
    AContext.Connection.IOHandler.InputBuffer.Clear;
  end;
end;


--
Remy Lebeau (TeamB)
Adam Hair

Posts: 18
Registered: 7/26/17
Re: Writing a Telnet Relay Application [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 13, 2017 5:36 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Hi Remy,

Thanks again for your reply, and my apologies. I'm using Delphi Seattle, so I am able to use Generics.

I've used your updated code, but am still having a problem where it appears that the data isn't being assigned to the queue. If I'm understanding the code correctly I think I have narrowed down the issue.

In the idTelnetDataAvailable event, I have put a debug break on the bolded line below:

list := IdTelnetServer1.Contexts.LockList;
try
for i := 0 to list.Count-1 do
begin
Ctx := TMyContext(TIdContext(list[i]));
try
Ctx.QueueLock.Enter;
try
Ctx.Queue.Enqueue(Buffer);
Ctx.QueueHasData.SetEvent;
finally
Ctx.QueueLock.Leave;
end;
except
end;
end;
finally
IdTelnetServer1.Contexts.UnlockList;
end;

When I view Buffer, I can see data, however after I execute this line, and I check Ctx.Queue - it returns no data being available as though no data has been added to the queue.

When I inspect buffer i get data such as:

(3, 2, 32, 32, 32, 53, 48, 46, 52, 48, 71, 3, 2, 32, 32, 32, 53, 48, 46, 52, 48, 71, 3, 2, 32, 32, 32, 53, 48, 46, 52, 48, 71, 3, 2, 32, 32, 32, 53, 48, 46, 52, 48, 71, 3, 2, 32, 32, 32, 53, 48, 46, 52, 48, 71, 3, 2, 32, 32, 32, 53, 48, 46, 52, 48, 71, 3, 2, 32, 32, 32, 53, 48, 46, 52, 48, 71, 3, 2, 32, 32, 32, 53, 48, 46, 52, 48, 71, 3, 2, 32, 32, 32, 53, 48, 46, 52, 48, 71, 3, 2, 32, 32, 32, 53, 48, 46, 52, 48, 71, 3, 2, 32, 32, 32, 53, 48, 46, 52, 48, 71, 3, 2, 32, 32, 32, 53, 48, 46, 52, 48, 71, 3, 2, 32, 32, 32, 53, 49, 46, 50, 48, 71, 3, 2, 32, 32, 32, 53, 49, 46, 50, 48, 71, 3, 2, 32, 32, 32, 53, 49, 46, 50, 48, 71)

When I inspect ctx.queue I get:
((), (), (nil,nil), 0)

ctx.queue.count = 0

it's as though the Enqueue code hasn't added any data, and because .count = 0 no data is ever transmitted on the server's event.

Thanks & Regards

Adam.

Edited by: Adam Hair on Aug 13, 2017 5:37 PM

Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Writing a Telnet Relay Application [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 14, 2017 10:27 AM   in response to: Adam Hair in response to: Adam Hair
Adam Hair wrote:

In the idTelnetDataAvailable event, I have put a debug break on the
bolded line below:
<snip>
When I view Buffer, I can see data, however after I execute this
line, and I check Ctx.Queue - it returns no data being available as
though no data has been added to the queue.

TQueue.Enqueue() is guaranteed to add the new Buffer to the Queue.

The only possible way I can think of for the behavior you have
described to happen is if another thread is waiting on a client's
QueueHasData event and gets woken up by the SetEvent() method, then a
task switch to that thread occurs and it accesses and clears the Queue
while the posting thread still has the client's QueueLock locked.

That should not be possible given the code I have given you, as all
threads that touch a client's Queue are supposed to lock the QueueLock
before accessing the Queue, thus multiple threads cannot access a
client's Queue at the same time.

Here is a safer version of the code to better enforce that locking:

uses
  ...,
  System.Generics.Collections, System.SyncObjs;
 
type
  TMyContext = class(TIdTelnetServerContext)
  private
    Queue: TQueue<TIdBytes>;
    QueueLock: TCriticalSection;
    QueueHasData: TEvent;
  public
    constructor Create(AConnection: TIdTCPConnection; AYarn: TIdYarn; 
AList: TIdContextThreadList = nil); override;
    destructor Destroy; override;
    procedure AddToQueue(const Buffer: TIdBytes);
    function ExtractQueue(ATimeout: Integer = 0): TArray<TIdBytes>;
  end;
 
...
 
constructor TMyContext.Create(AConnection: TIdTCPConnection; AYarn:
TIdYarn; AList: TIdContextThreadList = nil);
begin
  inherited;
  Queue := TQueue<TIdBytes>.Create;
  QueueLock := TCriticalSection.Create;
  QueueHasData := TEvent.Create(nil, True, False, '');
end;
 
destructor TMyContext.Destroy;
begin
  Queue.Free;
  QueueLock.Free;
  QueueHasData.Free;
  inherited;
end;
 
procedure TMyContext.AddToQueue(const Buffer: TIdBytes);
begin
  QueueLock.Enter;
  try
    Queue.Enqueue(Buffer);
    if Queue.Count = 1 then
      QueueHasData.SetEvent;
  finally
    QueueLock.Leave;
  end;
end;
 
function TMyContext.ExtractQueue(ATimeout: Integer = 0):
TArray<TIdBytes>;
begin
  Result := nil;
  if QueueHasData.WaitFor(ATimeout) = wrSignaled then
  begin
    QueueLock.Enter;
    try
      Result := Queue.ToArray;
      Queue.Clear;
    finally
      if Queue.Count = 0 then
        QueueHasData.ResetEvent;
      QueueLock.Leave;
    end;
  end;
end;
 
procedure TMainForm.FormCreate(Sender: TObject);
begin
  // set this before acticating the server...
  IdTelnetServer1.ContextClass := TMyContext;
end;
 
procedure TMainForm.IdTelnet1DataAvailable(Sender: TIdTelnet; const
Buffer: TIdBytes);
var
  list: TIdContextList;
  i : integer;
begin
  list := IdTelnetServer1.Contexts.LockList;
  try
    for i := 0 to list.Count-1 do
    begin
      try
        TMyContext(TIdContext(list[i])).AddToQueue(Buffer);
      except
      end;
    end;
  finally
    IdTelnetServer1.Contexts.UnlockList;
  end;
end;
 
procedure TMainForm.IdTelnetServer1Execute(AContext: TIdContext);
var
  Ctx: TMyContext;
  arr: TArray<TIdBytes>;
  i : integer;
begin
  Ctx := TMyContext(AContext);
 
  arr := Ctx.ExtractQueue;
  if arr <> nil then
  begin
    for i := Low(arr) to High(arr) do
      AContext.Connection.IOHandler.Write(arr[i]);
  end;
 
  if AContext.Connection.IOHandler.InputBufferIsEmpty then
  begin
    AContext.Connection.IOHandler.CheckForDataOnSource(0);
    AContext.Connection.IOHandler.CheckForDisconnect;
  end;
 
  if not AContext.Connection.IOHandler.InputBufferIsEmpty then
  begin
    // handle inbound client data/command as needed...
    AContext.Connection.IOHandler.InputBuffer.Clear;
  end;
end;


--
Remy Lebeau (TeamB)
Adam Hair

Posts: 18
Registered: 7/26/17
Re: Writing a Telnet Relay Application [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 14, 2017 5:44 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Good Morning Remy,

Thanks again for your reply. I have tried the new approach and seem to have the same issue. It's strange. My project has nothing else in it. (It's a blank form, with the code you've provided plus a button to activate the components on the form - nothing else).

I've put a debug on the TMyContext.AddToQueue event and have observed the following behaviour:

procedure TMyContext.AddToQueue(const Buffer: TIdBytes);
begin
QueueLock.Enter;
try
Queue.Enqueue(Buffer);
if Queue.Count = 1 then
QueueHasData.SetEvent;
finally
QueueLock.Leave;
end;
end;

At this point Queue.Count = 0, even though Length(Buffer) reports > 5 (in some test instances I was getting as high as 220)

I have placed a copy of my project, as well as a test transmitter I'm using to try and transmit at http://www.woodwindsys.com.au/tmp/indyrelayer.zip

I am using Delphi Seattle, with Indy 10.6.2.5311.

Do you have any other thoughts as to what the issue might be? (I have disabled antivirus and firewall on my machine just incase it was something as silly as that but to no avail).

From my initial observation it appears as though TQueue can't handle TidBytes.

Thanks & Regards

Adam.
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Writing a Telnet Relay Application [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 15, 2017 9:55 AM   in response to: Adam Hair in response to: Adam Hair
Adam Hair wrote:

Queue.Enqueue(Buffer);
if Queue.Count = 1 then

At this point Queue.Count = 0, even though Length(Buffer) reports > 5
(in some test instances I was getting as high as 220)

That is physically impossible, as Enqueue() is guaranteed to increment
the Count:

procedure TQueue<T>.Enqueue(const Value: T);
begin
  if Count = Length(FItems) then
    Grow;
  FItems[FHead] := Value;
  FHead := (FHead + 1) mod Length(FItems);
  Inc(FCount); // <-- incremented here
  Notify(Value, cnAdded); // <-- just fires the OnNotify event if any
end;


As I said earlier, the ONLY way the Count could remain 0 is if another
thread is getting into the Queue prematurely (ignoring the lock) and
clearing the Queue at the wrong time, which should be impossible given
the updated code. You need to keep debugging.

I have placed a copy of my project, as well as a test transmitter I'm
using to try and transmit at
http://www.woodwindsys.com.au/tmp/indyrelayer.zip

Using that exact code, I can't reproduce your issue. The Queue.Count
is never 0 after Enqueue(), and all threads respect the locking.

On a side note: in this code in the test server:

procedure TForm1.IdTelnetServer1Execute(AContext: TIdContext);
var
 s : String;
begin
 s := inttostr(10+random(80))+' '+#01#02#00;
 acontext.Connection.IOHandler.Write(ToBytes(s), length(tobytes(s)));
 sleep(20);
end;


You should'nt be calling ToBytes() multiple times like that. Call it
only once instead:

procedure TForm1.IdTelnetServer1Execute(AContext: TIdContext);
var
  s : String;
  bytes: TIdBytes;
begin
  s := IntToStr(10+Random(80))+' '+#01#02#00;
  bytes := ToBytes(s);
  AContext.Connection.IOHandler.Write(bytes);
  sleep(20);
end;


Or, simply don't use a TIdBytes at all, send the string as-is:

procedure TForm1.IdTelnetServer1Execute(AContext: TIdContext);
var
  s : String;
begin
  s := IntToStr(10+Random(80))+' '+#01#02#00;
  AContext.Connection.IOHandler.Write(s);
  sleep(20);
end;


That is just an observation. not the cause of your issues. Another
observation would be to move your TMyContext class declaration to the
unit's 'implmentation' section, as it doesn't belong in the 'interface'
section.

I am using Delphi Seattle, with Indy 10.6.2.5311.

The current version of Indy is 10.6.2.5432. You should consider
upgrading.

--
Remy Lebeau (TeamB)
Adam Hair

Posts: 18
Registered: 7/26/17
Re: Writing a Telnet Relay Application [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 15, 2017 6:53 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Hi Remy,

Thanks for your consistent help and patience.

I'm running the exact same project that I gave you to compare. I haven't snuck in any other code with threads or otherwise, so I can't see how there's another thread in my app (given it's so small and the same as what I gave you) that could be getting in and doing something to clear the queue.

I'm running Seattle Subscription Update 1, and I'm now suspicious that it might be a bug in Seattle, as I have now just tried running the same code in Delphi XE2 (with a minor change to get it to compile - I had to change the TidContextList back to a standard TList), and the same project works just fine as it should for me in XE2!

So - at least I can get around the issue for the time being by compiling in XE2 instead, so I'll just work in XE2 instead of Seattle.

(I'd be interested to know if you were running Seattle, or another IDE to see if that's the difference).

Thanks for the tips on the toBytes call as well! I really appreciate the time you've taken out to help me.

I did try updating Indy in Seattle to see if that would help, but somehow I've gone backwards - it's now reflecting version 10.6.2.0. (I updated by using SVN to obtain files from https://svn.atozed.com:444/svn/Indy10 which I believe is correct)?

I also tried debugging in Seattle to see what might be going on, but I can't seem to be able to debug into TQueue<t>.Enqueue either.

When I open up system.generics.collections and put breakpoints on lines in there it doesn't break, and I'm unable to F7 / trace into on the .Enqueue line of my application to see what occurs - it just skips straight to the next line in my code, with no increment on the count.

I've tried a different test app to see how TQueue works with strings (outside of indy) and it behaves as expected:

var
 Queue : TQueue<string>;
begin
 Queue := TQueue<String>.Create;
 memo1.lines.clear;
 memo1.lines.add('Count is '+inttostr(queue.Count));
 queue.Enqueue('First Line');
 memo1.lines.add('Count is '+inttostr(queue.Count));
 queue.Enqueue('Second Line');
 memo1.lines.add('Count is '+inttostr(queue.Count));


Interestingly enough I can F7 trace on the Enqueue in this test case, but it takes me to a different procedure (procedure TQueueHelper.InternalEnqueueString ), and not procedure TQueue<T>.Enqueue(const Value: T);

Either way - XE2 seems to be doing right by me at the moment, so I'll develop this app in XE2 instead as a work around.

Once again, thanks for your help and assistance with this.

Best Regards

Adam.
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Writing a Telnet Relay Application [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 15, 2017 8:49 PM   in response to: Adam Hair in response to: Adam Hair
Adam Hair wrote:

as I have now just tried running the same code in Delphi XE2 (with a
minor change to get it to compile - I had to change the TidContextList
back to a standard TList)

Then you are not using an up-to-date version of Indy.

(I'd be interested to know if you were running Seattle, or another
IDE to see if that's the difference).

I use XE2.

I did try updating Indy in Seattle to see if that would help, but
somehow I've gone backwards - it's now reflecting version 10.6.2.0.
(I updated by using SVN to obtain files from
https://svn.atozed.com:444/svn/Indy10 which I believe is correct)?

The version number that is in the code on SVN is indeed 10.6.2.0.
There is a TortoiseSVN script that updates the version number after the
code is pulled down from SVN. This is explained on Indy's website:

http://www.indyproject.org/Sockets/Download/svn.aspx

For Indy 10: if you use Tortoise as your SVN client, there are
StoreRevNum.bat and StoreRevNumHooks.bat scripts in the root \Lib
folder which can be used to update IdVers.inc and various .rc files
with the current SVN revision number prior to compiling.
StoreRevNum.bat invokes Tortoise's command-line SubWCRev.exe utility
and thus can be run at any time, whereas StoreRevNumHooks.bar is meant
to be used only with Tortoise's Post-Commit and Post-Update hooks.
Before either .bat file can be used, you need to manually edit the .bat
files and update the IndyLib environment variable to point at the path
to the working copy of your local Indy 10 \Lib source folder.

The nightly zipped snapshot for Indy 10 already has the SVN revision
number applied to its files.

I also tried debugging in Seattle to see what might be going on, but
I can't seem to be able to debug into TQueue<t>.Enqueue either.

Did you enable the "Use debug dcus" option in your project settings?

Interestingly enough I can F7 trace on the Enqueue in this test case,
but it takes me to a different procedure (procedure
TQueueHelper.InternalEnqueueString ), and not procedure
TQueue<T>.Enqueue(const Value: T);

It starts out in TQueue<T>.Enqueue(), but the way the code is inlined
at compile-time, it ends up jumping straight into
TQueueHelper.InternalEnqueueString() when T is String.

Looking at TQueue in Seattle, it has a MUCH different implementation
than TQueue in XE2. I don't know why Embarcadero did what they did,
but it appears they forgot to include support for dynamic arrays in the
new implementation. Feel free to file a bug report about that.

In which case, I would suggest replacing TQueue<T> with TList<T>
instead, which offhand appears to have support for dynamic arrays:

uses
  ...,
  System.Generics.Collections, System.SyncObjs;
 
type
  TMyContext = class(TIdTelnetServerContext)
  private
    Queue: TList<TIdBytes>;
    QueueLock: TCriticalSection;
    QueueHasData: TEvent;
  public
    constructor Create(AConnection: TIdTCPConnection; AYarn: TIdYarn; 
AList: TIdContextThreadList = nil); override;
    destructor Destroy; override;
    procedure AddToQueue(const Buffer: TIdBytes);
    function ExtractQueue(ATimeout: Integer = 0): TArray<TIdBytes>;
  end;
 
...
 
constructor TMyContext.Create(AConnection: TIdTCPConnection; AYarn:
TIdYarn; AList: TIdContextThreadList = nil);
begin
  inherited;
  Queue := TList<TIdBytes>.Create;
  QueueLock := TCriticalSection.Create;
  QueueHasData := TEvent.Create(nil, True, False, '');
end;
 
destructor TMyContext.Destroy;
begin
  Queue.Free;
  QueueLock.Free;
  QueueHasData.Free;
  inherited;
end;
 
procedure TMyContext.AddToQueue(const Buffer: TIdBytes);
begin
  QueueLock.Enter;
  try
    Queue.Add(Buffer);
    if Queue.Count = 1 then
      QueueHasData.SetEvent;
  finally
    QueueLock.Leave;
  end;
end;
 
function TMyContext.ExtractQueue(ATimeout: Integer = 0):
TArray<TIdBytes>;
begin
  Result := nil;
  if QueueHasData.WaitFor(ATimeout) = wrSignaled then
  begin
    QueueLock.Enter;
    try
      Result := Queue.ToArray;
      Queue.Clear;
    finally
      if Queue.Count = 0 then
        QueueHasData.ResetEvent;
      QueueLock.Leave;
    end;
  end;
end;
 
procedure TMainForm.FormCreate(Sender: TObject);
begin
  // set this before acticating the server...
  IdTelnetServer1.ContextClass := TMyContext;
end;
 
procedure TMainForm.IdTelnet1DataAvailable(Sender: TIdTelnet; const
Buffer: TIdBytes);
var
  list: TIdContextList;
  i : integer;
begin
  list := IdTelnetServer1.Contexts.LockList;
  try
    for i := 0 to list.Count-1 do
    begin
      try
        TMyContext(TIdContext(list[i])).AddToQueue(Buffer);
      except
      end;
    end;
  finally
    IdTelnetServer1.Contexts.UnlockList;
  end;
end;
 
procedure TMainForm.IdTelnetServer1Execute(AContext: TIdContext);
var
  Ctx: TMyContext;
  arr: TArray<TIdBytes>;
  i : integer;
begin
  Ctx := TMyContext(AContext);
 
  arr := Ctx.ExtractQueue;
  if arr <> nil then
  begin
    for i := Low(arr) to High(arr) do
      AContext.Connection.IOHandler.Write(arr[i]);
  end;
 
  if AContext.Connection.IOHandler.InputBufferIsEmpty then
  begin
    AContext.Connection.IOHandler.CheckForDataOnSource(0);
    AContext.Connection.IOHandler.CheckForDisconnect;
  end;
 
  if not AContext.Connection.IOHandler.InputBufferIsEmpty then
  begin
    // handle inbound client data/command as needed...
    AContext.Connection.IOHandler.InputBuffer.Clear;
  end;
end;


--
Remy Lebeau (TeamB)
Adam Hair

Posts: 18
Registered: 7/26/17
Re: Writing a Telnet Relay Application [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 15, 2017 11:16 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Good Afternoon Remy,

Thanks again for your reply and persistent and the additional information.

The version number that is in the code on SVN is indeed 10.6.2.0.
There is a TortoiseSVN script that updates the version number after the
code is pulled down from SVN. This is explained on Indy's website:

http://www.indyproject.org/Sockets/Download/svn.aspx

Aah - thank you.

I also tried debugging in Seattle to see what might be going on, but
I can't seem to be able to debug into TQueue<t>.Enqueue either.

Did you enable the "Use debug dcus" option in your project settings?

Yes - Use Debug DCU's were in my project - not sure if there are any compiler directives in those units that exclude that unit from being included or not.

Interestingly enough I can F7 trace on the Enqueue in this test case,
but it takes me to a different procedure (procedure
TQueueHelper.InternalEnqueueString ), and not procedure
TQueue<T>.Enqueue(const Value: T);

It starts out in TQueue<T>.Enqueue(), but the way the code is inlined
at compile-time, it ends up jumping straight into
TQueueHelper.InternalEnqueueString() when T is String.

That explains a bit - thanks.

Looking at TQueue in Seattle, it has a MUCH different implementation
than TQueue in XE2. I don't know why Embarcadero did what they did,
but it appears they forgot to include support for dynamic arrays in the
new implementation. Feel free to file a bug report about that.

Does Embarcadero actually do bug fixes for older versions of Dephi still? (I was under the impression that they just 'fix' it in the next release to get users to upgrade again for the fixes)?

In which case, I would suggest replacing TQueue<T> with TList<T>
instead, which offhand appears to have support for dynamic arrays:

Excellent - thank you! I've done this in Seattle, and it works! :-)

Thanks once again for your persistence in getting this solved!

Best Regards

Adam.
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Writing a Telnet Relay Application [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 16, 2017 11:28 AM   in response to: Adam Hair in response to: Adam Hair
Adam Hair wrote:

Does Embarcadero actually do bug fixes for older versions of Dephi
still?

I don't know. My understanding was that under the new subscription
model, bug fixes were supposed to be backported for the previous 2
versions (which would include Seattle and Berlin right now). But I
can't find any information about that, so I don't know if they are
still doing it. And I don't have Berlin or Tokyo installed to check if
the bug has already been fixed or not. Worse case, you file a bug
report for Seattle and see if it gets closed as "fixed" or "won't do".

--
Remy Lebeau (TeamB)
Adam Hair

Posts: 18
Registered: 7/26/17
Re: Writing a Telnet Relay Application [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 16, 2017 3:25 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:
Adam Hair wrote:

Does Embarcadero actually do bug fixes for older versions of Dephi
still?

I don't know. My understanding was that under the new subscription
model, bug fixes were supposed to be backported for the previous 2
versions (which would include Seattle and Berlin right now). But I
can't find any information about that, so I don't know if they are
still doing it. And I don't have Berlin or Tokyo installed to check if
the bug has already been fixed or not. Worse case, you file a bug
report for Seattle and see if it gets closed as "fixed" or "won't do".

Well, there's only one way to find out. :-)

I've created a new bug report - RSP-18861

You must have thought I was going crazy when I was saying that Queue wasn't incrementing. :-) (I was wondering if I was going crazy myself).

Thanks once more for your help!
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Writing a Telnet Relay Application [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 16, 2017 6:08 PM   in response to: Adam Hair in response to: Adam Hair
Adam Hair wrote:

I've created a new bug report - RSP-18861

I updated your ticket to make its wording be less Indy-centric. It would have more chance of getting fixed if you also update the example code to remove the dependency on Indy, by replacing TIdBytes with the RTL's own TBytes or TArray<Bytes> type, and ToBytes() with TEncoding.GetBytes().

--
Remy Lebeau (TeamB)
Adam Hair

Posts: 18
Registered: 7/26/17
Re: Writing a Telnet Relay Application [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 16, 2017 7:09 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:
I updated your ticket to make its wording be less Indy-centric. It would have more chance of getting fixed if you also update the example code to remove the dependency on Indy, by replacing TIdBytes with the RTL's own TBytes or TArray<Bytes> type, and ToBytes() with TEncoding.GetBytes().

Thanks Remy! I appreciate the updates you have made. I've modified the example to remove Indy from the equation.

Cheers

Adam
Achim Strauch

Posts: 13
Registered: 4/12/02
Re: Writing a Telnet Relay Application [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 17, 2017 1:07 AM   in response to: Adam Hair in response to: Adam Hair
Adam,

Adam Hair wrote:


Thanks Remy! I appreciate the updates you have made. I've modified the example to remove Indy from the equation.

I checked your example in 10.1.2 and 10.2. The count was 1 and the data was added.

But there are other errors (RSP-17728). TQueue with dynamic arrays is not usable, TList is ok for me.

--
Achim Strauch

Adam Hair

Posts: 18
Registered: 7/26/17
Re: Writing a Telnet Relay Application [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 17, 2017 4:09 PM   in response to: Achim Strauch in response to: Achim Strauch
Achim Strauch wrote:
I checked your example in 10.1.2 and 10.2. The count was 1 and the data was added.

But there are other errors (RSP-17728). TQueue with dynamic arrays is not usable, TList is ok for me.

Thanks Achim, At this stage I'll leave my development to XE2 for this project, and will avoid TQueue's in my development (or use the workaround provided in RSP-17728 if required).
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Writing a Telnet Relay Application [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 17, 2017 10:26 AM   in response to: Adam Hair in response to: Adam Hair
Adam Hair wrote:

Thanks Remy! I appreciate the updates you have made. I've modified
the example to remove Indy from the equation.

Turns out your ticket is a duplicate of RSP-13196 (which contains a
workaround). I've posted a comment on your ticket about that.

--
Remy Lebeau (TeamB)
Adam Hair

Posts: 18
Registered: 7/26/17
Re: Writing a Telnet Relay Application [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 17, 2017 4:08 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:
Turns out your ticket is a duplicate of RSP-13196 (which contains a
workaround). I've posted a comment on your ticket about that.

Thanks Remy. Looking at RSP-13196 it looks like they've chosen to mark the issue as 'Fixed' even though Seattle doesn't have it resolved. Thankfully there is a workaround - I appreciate the heads up.

Cheers

Adam.
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Writing a Telnet Relay Application [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 17, 2017 9:11 PM   in response to: Adam Hair in response to: Adam Hair
Adam Hair wrote:

Thanks Remy. Looking at RSP-13196 it looks like they've chosen to
mark the issue as 'Fixed' even though Seattle doesn't have it
resolved.

Because it was fixed in Berlin instead, not in Seattle.

--
Remy Lebeau (TeamB)
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Writing a Telnet Relay Application [Edit] [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 15, 2017 9:56 AM   in response to: Adam Hair in response to: Adam Hair
Adam Hair wrote:

From my initial observation it appears as though TQueue can't handle
TidBytes.

TQueue handles TIdBytes just fine.

--
Remy Lebeau (TeamB)
Adam Hair

Posts: 18
Registered: 7/26/17
Re: Writing a Telnet Relay Application [Edit] [Edit] [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 15, 2017 11:17 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:
Adam Hair wrote:

From my initial observation it appears as though TQueue can't handle
TidBytes.

TQueue handles TIdBytes just fine.

... Just not in Delphi Seattle. ;-)
Shlomo Abuisak

Posts: 100
Registered: 9/18/10
Re: Writing a Telnet Relay Application
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 10, 2017 12:02 PM   in response to: Adam Hair in response to: Adam Hair
On 10/08/2017 04:07, Adam Hair wrote:
Hi,

I have a situation where I wish to connect to a device using Telnet to read data. The device only allows a single connection, however we are wanting multiple users to be able to read simultaneously. I'm therefore looking at creating a relay application that will connect to the server using TidTelnet, and then relay the messages to any connections using TidTelnetServer.

However I am having some problems trying to get this to work. (I'm not very cluey when it comes to indy components or winsock at all for that matter).

I can establish the connection with the TidTelnet client using the TidTelnet.DataAvailable event.

However I'm not sure how to relay this through to the tIdTelnetServer component.

This is where I've gotten to so far - but I know that this is wrong. (Because it's not working).

procedure TMainForm.IdTelnet1DataAvailable(Sender: TIdTelnet; const Buffer: TIdBytes);
var
i : integer;
AContext: TIdContext;
begin
for I := 0 to IdTelnetServer1.Contexts.count - 1 do
begin
AContext := TidContext(IDTelnetServer1.Contexts.LockList.items[i]);
AContext.Connection.IOHandler.Write(Buffer);
end;
end;

Can someone please point me in the right direction?

(This only needs to have unidirectional communication - there's no need for clients to be able to transmit back to the original device).

Thanks & Regards

Adam.

If i understood you i made such an application
years ago to connect with Telnet to many servers and send messages.
It is used in a very large company.
It is a big project not that simple.
I do not know how to help.
May be sell ?
In any case my email is
limelect at gmail dot com
or www dot limelect dot com
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02