Login or Sign Up to become a member!
LessThanDot Site Logo

LessThanDot

A Technical Community for IT Professionals

Less Than Dot is a community of passionate IT professionals and enthusiasts dedicated to sharing technical knowledge, experience, and assistance. Inside you will find reference materials, interesting technical discussions, and expert tips and commentary. Once you register for an account you will have immediate access to the forums and all past articles and commentaries.

LTD Social Sitings

Lessthandot twitter Lessthandot Linkedin Lessthandot facebook Lessthandot rss

Note: Watch for social icons on posts by your favorite authors to follow their postings on these and other social sites.

Highly Rated Users

Forum
No Posts Rated

Top 50
Given
Received

Links

Wiki
Blog

Forum Statistics

Users
Members:
1873
Members Online:
1
Guests Online:
84

Total Post History
Posts:
81445
Topics:
18714

7-Day Post History
New Posts:
0
New Topics:
0
Active Topics:
0

Our newest member
BitogClogs

Other

FAQ
All times are UTC [ DST ]

Google Ads

Solution for out of memory when resizing an array

Please wait...

Solution for out of memory when resizing an array

Postby Emtucifor on Mon Jul 07, 2008 9:09 pm

So you're trying to do something simple:

  1. Dim Vals() as Long
  2. Redim Vals(67108863)
  3. ' Run-time error '7': Out of memory

If you absolutely must have this working, then write a wrapper class that does it for you in chunks.

  1. 'Standard Module:
  2. Dim Longs As CSegmentedLongArray
  3.  
  4. Sub Main()
  5.    Set Longs = New CSegmentedLongArray
  6.    Longs.Size = 64000000
  7.    Longs(63999999) = 123
  8.    Debug.Print Longs(63999999)
  9. End Sub

  1. 'Class Module named "CSegmentedLongArray"
  2. Option Explicit
  3.  
  4. Private Type TLongArraySegment
  5.    LongArray() As Long
  6.    StartIndex As Long
  7.    EndIndex As Long
  8. End Type
  9.    
  10. Private LongArrays() As TLongArraySegment
  11. Private LongArrayCount As Long
  12. Private LongCount As Long
  13. Private MaxSizeAllocated As Long
  14.  
  15. Public Property Let Size(ByVal TheSize As Long)
  16.    Dim SizeToAllocate As Long
  17.    Dim Success As Boolean
  18.    Dim Index As Long
  19.    Dim Temp() As Long
  20.    
  21.    If TheSize <= 0 Then Err.Raise 50001, "CSegmentedLongArray.Size", "The supplied size was out of range."
  22.    
  23.    If MaxSizeAllocated = 0 Then
  24.       SizeToAllocate = TheSize
  25.    Else
  26.       SizeToAllocate = MaxSizeAllocated
  27.    End If
  28.    
  29.    ReDim LongArrays(0) ' if it was already created, all data is lost
  30.    LongArrayCount = 0
  31.    LongCount = TheSize
  32.    
  33.    On Error GoTo Size_Error
  34.    Success = True
  35.    Do While TheSize > 0 And Success = True
  36.       LongArrayCount = LongArrayCount + 1
  37.       ReDim Preserve LongArrays(LongArrayCount - 1)
  38.       If TheSize < SizeToAllocate Then SizeToAllocate = TheSize
  39.       With LongArrays(LongArrayCount - 1)
  40.          Do
  41.             Success = True
  42.             ReDim .LongArray(SizeToAllocate)
  43.             If Success = True Then
  44.                If SizeToAllocate > MaxSizeAllocated Then MaxSizeAllocated = SizeToAllocate ' once successful, use this as the starting point in the future
  45.                TheSize = TheSize - SizeToAllocate
  46.                .StartIndex = Index
  47.                .EndIndex = Index + SizeToAllocate - 1
  48.                Index = Index + SizeToAllocate
  49.             End If
  50.          Loop Until Success
  51.          End With
  52.    Loop
  53.    
  54. Size_Exit:
  55.    Exit Property
  56.  
  57. Size_Error:
  58.    If Err.Number = 7 Then ' Out of Memory
  59.       If SizeToAllocate > 1048576 Then
  60.          SizeToAllocate = SizeToAllocate - 1048576 'reduce the attempted allocation size by 1 MB at a time.
  61.          Success = False
  62.          Resume Next
  63.       Else
  64.          Err.Raise 50002, "CSegmentedLongArray.Size", "Out of memory attempting to allocate long array of size " & TheSize & "."
  65.          GoTo Size_Exit
  66.       End If
  67.    End If
  68.    On Error GoTo 0
  69.    Resume
  70. End Property
  71.  
  72. Public Property Get Size() As Long
  73.    Size = LongCount
  74. End Property
  75.  
  76. Public Property Let Item(Index As Long, Value As Long)
  77.    If Index < 0 Or Index >= LongCount Then Err.Raise 50000, "CLongArray.Item.Let", "The supplied index was out of range."
  78.    Dim LongArrayNum As Long
  79.    For LongArrayNum = 0 To LongArrayCount - 1
  80.       If LongArrays(LongArrayNum).StartIndex <= Index And LongArrays(LongArrayNum).EndIndex >= Index Then
  81.          LongArrays(LongArrayNum).LongArray(Index - LongArrays(LongArrayNum).StartIndex) = Value
  82.       End If
  83.    Next
  84. End Property
  85.  
  86. Public Property Get Item(Index As Long) As Long
  87.    If Index < 0 Or Index >= LongCount Then Err.Raise 50000, "CLongArray.Item.Get", "The supplied index was out of range."
  88.    Dim LongArrayNum As Long
  89.    For LongArrayNum = 0 To LongArrayCount - 1
  90.       If LongArrays(LongArrayNum).StartIndex <= Index And LongArrays(LongArrayNum).EndIndex >= Index Then
  91.          Item = LongArrays(LongArrayNum).LongArray(Index - LongArrays(LongArrayNum).StartIndex)
  92.       End If
  93.    Next
  94. End Property

Put the cursor on one of the Item properties and click Tools -> Procedure Attributes -> Advanced and change the Procedure ID to "(Default)." This makes the Item property work without having to say Longs.Item(123), you can just do Longs(123).

While this is undoubtedly going to be WAY slower than having a simple array of the required size, if you absolutely need it and can't get the memory this is one way to handle the problem. The idea can be extended to arrays of anything. In my own code I am not making arrays of long arrays but arrays of a class (that itself has long arrays inside of it). The class couldn't allocate an array large enough on my system, so I made a wrapper class that does it in segments, whose interface exactly matches the original bit array class's interface.
God cries a little bit every time someone builds a database.
User avatar
Emtucifor
Guru
Guru
LTD Gold - Rating: 1033LTD Gold - Rating: 1033LTD Gold - Rating: 1033LTD Gold - Rating: 1033LTD Gold - Rating: 1033
LTD Gold - Rating: 1033LTD Gold - Rating: 1033LTD Gold - Rating: 1033LTD Gold - Rating: 1033LTD Gold - Rating: 1033
LTD Gold - Rating: 1033
 
Posts: 2835
Joined: Fri May 30, 2008 9:30 pm
Location: Bellingham, WA
Unrated